Personality Analysis Using Machine Learning

Sponsor	Withheld
Team Name	Digital Profilers
Duration	Fall 2017 - Spring 2018
Faculty Advisors	Bruce Bolden
Students	Austin Sass Seth Forrest Paden Rumsey Lise Welch

The goal of this project is to develop a software capable of performing sentiment analysis on bodies of text with a given author. This tool will mimic IBM Watson's Personality Insights Service - a popular online resource and application programming interface (API) for personality classification.

Problem Definition

Background

There are many sentiment analysis tools available for public use, however, many have significant drawbacks, are not well documented, or perform classification with relatively poor accuracy. We plan to produce the initial framework needed to create a new sentiment analysis tool that remedies many of these problems. Specifically, we will focus on web mining Twitter tweets, storing tweets in a database with relevant statistical information, and training a machine from the collected data. In time, the results of this project may develop into a mobile app.

Requirements

Data/Web Mining Requirements
No.	Need	Metric
1	Text Sample	1200 - 3000 words
2	Quality of Text	Raw/Unprocessed
3	Author	Singular/Separable
4	Retain Information	Copyable to Repository
5	Semi-Automated	10000's tweets/crawl
6	Location Retention (i.e., the URL)	Scraper Pulls URL

Database Requirements
No.	Need	Metric
1	Text Storage	1,000,000's of entries
2	Links to local copy	Cataloged
3	Author of text	Cataloged
4	IBM Watson Output	Numerics/Classifications indexed

Trained Machine Requirements
No.	Need	Metric
1	Process entries	1,000,000's to train
2	Approximate IBM Watson Output	Mean Average Error

Deliverables

• Twitter mining tool capable of collecting and combining numerous tweets from verified users/authors into text samples
• Text samples passed through IBM Watson Personality Insights Service
• Database with large collection of authors, indexed IBM Watson output, and statistics
• Machine trained on collected data

Design

Database

For this project we are using a MySQL database. Each entry for the database contains the author’s name, the original text file, the URL of the source, the JSON objects from our scrapper (Tweepy), and the JSON objects from IBM Watson’s output. Eventually we will add the Five Factor Model information results from our machine. Until then we will collect data in order to use it to train our machine.

Twitter Scraping

Sample is based on Twitter user profiles. A Python package called Tweepywas used to extract data from Twitter. Twitter restricts the number of API calls that can be made, so the scraper is be set up to run without intervention. Tweepy API information can be found HERE

Scraper	Flow

Text Preprocessing

Input text will be processed using an algorithm called GloVe*[8]. GloVe uses a strategy in which relationships between words can be found by analyzing their co-occurrence. Specifically, GloVe optimizes a global log-bilinear language model with a weighted least squares objective such that the difference between the dot product and co-occurrence probability between any two word feature vectors is zero. The resulting trained word vectors can be used as features in various machine learning algorithms.

GloVe Model	Explanation
	Weighting function Prevents overemphasis of large co-occurrence counts Dot product Used to encode co-occurrence probabilities in vector space Biases Maintains symmetric properties and absorbs independent variables Log of co-occurrence count Interacts with the bias bi to produce the co-occurrence probability of j in the context of word i

Machine Learning

Two machine learning (ML) approaches, Gaussian Process (GP) and Convolution Neural Network (CNN), were considered. Comparisons were made between the two to determine which approach would best suit the needs of the project. Due to the statistical complexities of GP [9], it was decided a CNN would be implemented.

Approach	Pros	Cons
Gaussian Process	Uses GloVe output as features	Statistically heavy Difficult to determine best co-variance function
Convolution Neural Network	Used for five-factor model personality classification Identifies most relevant layers (performance)

A CNN personality classifier based on Majumder et al. [6] implemented by SenticNet was chosen. The package includes text pre-processing, which can take the place of GloVe. Test will have to be conducted to determine whether GloVe provides any meaningful benefit.

Training

The CNN was trained on Watson's output, which includes both percentile and raw scores for each personality trait. Mean absolute error was used as a metric to measure how far off our machine was from Watson. Below are the MAE values for each trait based on percentile and raw scores and data set sizes of 1000 and 6000 text samples. For comparison, IBM Watson achieved a MAE score of 0.12 averaged across all five traits.

Project Learning

References

[1] Adedoyin-Olowe, Mariam, et al. “A Survey of Data Mining Techniques for Social Media Analysis.” 2013, Journal of Data Mining \& Digital Humanities, 2014 (June 24, 2014) jdmdh:18.

[2] Arnoux, Pierre-Hadrien, et al. “25 Tweets to Know You: A New Model to Predict Personality with Social Media.” 2017. https://arxiv.org/ftp/arxiv/papers/1704/1704.05513.pdf

[3] Digman, John M. "Personality structure: Emergence of the five-factor model. Annual Review of Psychology" 1990. 41.1: 417–440.

[4] Kalghatgi, M. P., et al. “A Neural Network Approach to Personality Prediction based on the Big-Five Model.” International Journal of Innovative Research in Advanced Engineering, vol. 2, no. 8, 2015, pp. 56–63.

[5] Kaur, Arvinder, and Deepti Chopra. “Comparison of Text Mining Tools.” Institute of Electrical and Electronics Engineers, 7 Sept. 2016, pp. 186–192. ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7784950.

[6] Majumder, Navonil, et al. “Deep Learning-Based Document Modeling for Personality Detection from Text.” IEEE Intelligent Systems, vol. 32, no. 2, 2017, pp. 74–79., doi:10.1109/mis.2017.23.

[7] Mnih, Andriy, Hinton, Geoffrey. "Three New Graphical Models for Statistical Language Modelling." 24th International Conference on Machine Learning, 2007

[8] Pennington, Jeffrey, Socher, Richard, Manning D., Christoper. "GloVe: Global Vectors for Word Representation." 2014. https://nlp.stanford.edu/pubs/glove.pdf.

[9] Rasmussen E, Carl, Williams K. I., Christoper. "Gaussian Processes for Machine Learning." 2006. http://www.gaussianprocess.org/gpml/.

Important Links

Tweepy Documentation
Watson Personality Insights
GloVe & GitHub Repository
Gaussian Processes
WEKA
Intro to CNN's
Mairesse Feature-Based Personality Recognizer

Team Information

Picture	Bio	Discipline
	Paden Rumsey: I come from a small town in Idaho called St. Anthony. I came to the University of Idaho almost three and half years ago and I’m finally graduating with a B.S. in Computer Science next spring. I initially got into computer science because of my interest in artificial intelligence and “smart” computers. The trends appeared to be moving in that direction and it seemed like a field I would really enjoy. The classes I’ve taken at the university have only strengthened that conviction. This senior project is a great opportunity to learn more techniques in machine learning and display the ones I’ve already learned.	Computer Science
	Lise Welch: I was born and raised in Boise, Idaho and started attending the University of Idaho in the Fall of 2013. I dabbled in Mechanical and Electrical Engineering my first couple years at the university but have since found my passion in Computer Science and am looking forward to graduating with my B.S. in the Spring. Outside of school you can find me at the Alumni Office helping them plan the university’s biggest events such as Homecoming and Parents’ Weekend.	Computer Science
	Austin Sass: I've lived in the area of Moscow, Idaho for most of my life. Most of my time is currently being spent in the Neurophysiological Imaging and Modeling Laboratory at the University of Idaho, where myself and team members are studying Space Flight-Associated Neuro-ocular Syndrome as it pertains to visual impairment in Astronauts. On the side, I develop flight simulation software and play music with the "Sass Brothers" band.	Computer Science
	Seth Forrest: I am a Technophile, I love exploring the application of technology in our daily lives. I am studying Mechanical Engineering and Computer Science in order to further investigate how tech can be integrated into our culture. I have a particular passion for the the union of the cyber and the physical, and how technology can be used to improve people's lives.	Computer Science/Mechanical Engineering

Document Archive

• Meeting Minutes
• Final Design Review