Personality Analysis Using Machine Learning

From Mindworks
Jump to navigation Jump to search
Sponsor Withheld
Team Name Digital Profilers
Duration Fall 2017 - Spring 2018
Faculty Advisors
  • Bruce Bolden
Students
  • Austin Sass
  • Seth Forrest
  • Paden Rumsey
  • Lise Welch

The goal of this project is to develop a software capable of performing sentiment analysis on bodies of text with a given author. This tool will mimic IBM Watson's Personality Insights Service - a popular online resource and application programming interface (API) for personality classification.


Problem Definition[edit | edit source]

Background[edit | edit source]

There are many sentiment analysis tools available for public use, however, many have significant drawbacks, are not well documented, or perform classification with relatively poor accuracy. We plan to produce the initial framework needed to create a new sentiment analysis tool that remedies many of these problems. Specifically, we will focus on web mining Twitter tweets, storing tweets in a database with relevant statistical information, and training a machine from the collected data. In time, the results of this project may develop into a mobile app.

Requirements[edit | edit source]

Data/Web Mining Requirements
No. Need Metric
1 Text Sample 1200 - 3000 words
2 Quality of Text Raw/Unprocessed
3 Author Singular/Separable
4 Retain Information Copyable to Repository
5 Semi-Automated 10000's tweets/crawl
6 Location Retention (i.e., the URL) Scraper Pulls URL
Database Requirements
No. Need Metric
1 Text Storage 1,000,000's of entries
2 Links to local copy Cataloged
3 Author of text Cataloged
4 IBM Watson Output Numerics/Classifications indexed
Trained Machine Requirements
No. Need Metric
1 Process entries 1,000,000's to train
2 Approximate IBM Watson Output Mean Average Error

Deliverables[edit | edit source]

  • Twitter mining tool capable of collecting and combining numerous tweets from verified users/authors into text samples
  • Text samples passed through IBM Watson Personality Insights Service
  • Database with large collection of authors, indexed IBM Watson output, and statistics
  • Machine trained on collected data

Design[edit | edit source]

2017 PersonalityAnalysisUsingMachineLearning overview.png

Database[edit | edit source]

For this project we are using a MySQL database. Each entry for the database contains the author’s name, the original text file, the URL of the source, the JSON objects from our scrapper (Tweepy), and the JSON objects from IBM Watson’s output. Eventually we will add the Five Factor Model information results from our machine. Until then we will collect data in order to use it to train our machine.

Twitter Scraping[edit | edit source]

Sample is based on Twitter user profiles. A Python package called Tweepywas used to extract data from Twitter. Twitter restricts the number of API calls that can be made, so the scraper is be set up to run without intervention. Tweepy API information can be found HERE

Scraper Flow
2018 PersonalityAnalysisUsingMachineLearning Scraper.png
2018 PersonalityAnalysisUsingMachineLearning ScraperDiag.png

Text Preprocessing[edit | edit source]

Input text will be processed using an algorithm called GloVe*[8]. GloVe uses a strategy in which relationships between words can be found by analyzing their co-occurrence. Specifically, GloVe optimizes a global log-bilinear language model with a weighted least squares objective such that the difference between the dot product and co-occurrence probability between any two word feature vectors is zero. The resulting trained word vectors can be used as features in various machine learning algorithms.

GloVe Model Explanation
2018 Personality Analysis Using Machine Learning global model.JPG
Weighting function
Prevents overemphasis of large co-occurrence counts
Dot product
Used to encode co-occurrence probabilities in vector space
Biases
Maintains symmetric properties and absorbs independent variables
Log of co-occurrence count
Interacts with the bias bi to produce the co-occurrence probability of j in the context of word i

Machine Learning[edit | edit source]

Two machine learning (ML) approaches, Gaussian Process (GP) and Convolution Neural Network (CNN), were considered. Comparisons were made between the two to determine which approach would best suit the needs of the project. Due to the statistical complexities of GP [9], it was decided a CNN would be implemented.


Approach Pros Cons
Gaussian Process
  • Uses GloVe output as features
  • Statistically heavy
  • Difficult to determine best co-variance function
Convolution Neural Network
  • Used for five-factor model personality classification
  • Identifies most relevant layers (performance)

A CNN personality classifier based on Majumder et al. [6] implemented by SenticNet was chosen. The package includes text pre-processing, which can take the place of GloVe. Test will have to be conducted to determine whether GloVe provides any meaningful benefit.

Training[edit | edit source]

The CNN was trained on Watson's output, which includes both percentile and raw scores for each personality trait. Mean absolute error was used as a metric to measure how far off our machine was from Watson. Below are the MAE values for each trait based on percentile and raw scores and data set sizes of 1000 and 6000 text samples. For comparison, IBM Watson achieved a MAE score of 0.12 averaged across all five traits.

2018 Personality Analysis Using Machine Learning MAE.PNG

Project Learning[edit | edit source]

References[edit | edit source]

[1] Adedoyin-Olowe, Mariam, et al. “A Survey of Data Mining Techniques for Social Media Analysis.” 2013, Journal of Data Mining \& Digital Humanities, 2014 (June 24, 2014) jdmdh:18.

[2] Arnoux, Pierre-Hadrien, et al. “25 Tweets to Know You: A New Model to Predict Personality with Social Media.” 2017. https://arxiv.org/ftp/arxiv/papers/1704/1704.05513.pdf

[3] Digman, John M. "Personality structure: Emergence of the five-factor model. Annual Review of Psychology" 1990. 41.1: 417–440.

[4] Kalghatgi, M. P., et al. “A Neural Network Approach to Personality Prediction based on the Big-Five Model.” International Journal of Innovative Research in Advanced Engineering, vol. 2, no. 8, 2015, pp. 56–63.

[5] Kaur, Arvinder, and Deepti Chopra. “Comparison of Text Mining Tools.” Institute of Electrical and Electronics Engineers, 7 Sept. 2016, pp. 186–192. ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7784950.

[6] Majumder, Navonil, et al. “Deep Learning-Based Document Modeling for Personality Detection from Text.” IEEE Intelligent Systems, vol. 32, no. 2, 2017, pp. 74–79., doi:10.1109/mis.2017.23.

[7] Mnih, Andriy, Hinton, Geoffrey. "Three New Graphical Models for Statistical Language Modelling." 24th International Conference on Machine Learning, 2007

[8] Pennington, Jeffrey, Socher, Richard, Manning D., Christoper. "GloVe: Global Vectors for Word Representation." 2014. https://nlp.stanford.edu/pubs/glove.pdf.

[9] Rasmussen E, Carl, Williams K. I., Christoper. "Gaussian Processes for Machine Learning." 2006. http://www.gaussianprocess.org/gpml/.

Important Links[edit | edit source]

Tweepy Documentation
Watson Personality Insights
GloVe & GitHub Repository
Gaussian Processes
WEKA
Intro to CNN's
Mairesse Feature-Based Personality Recognizer

Team Information[edit | edit source]

Picture Bio Discipline
Paden Rumsey:

I come from a small town in Idaho called St. Anthony. I came to the University of Idaho almost three and half years ago and I’m finally graduating with a B.S. in Computer Science next spring. I initially got into computer science because of my interest in artificial intelligence and “smart” computers. The trends appeared to be moving in that direction and it seemed like a field I would really enjoy. The classes I’ve taken at the university have only strengthened that conviction. This senior project is a great opportunity to learn more techniques in machine learning and display the ones I’ve already learned.

Computer Science
Lise Welch:

I was born and raised in Boise, Idaho and started attending the University of Idaho in the Fall of 2013. I dabbled in Mechanical and Electrical Engineering my first couple years at the university but have since found my passion in Computer Science and am looking forward to graduating with my B.S. in the Spring. Outside of school you can find me at the Alumni Office helping them plan the university’s biggest events such as Homecoming and Parents’ Weekend.

Computer Science
2017 PersonalityAnalysisUsingMachineLearning AustinSassPortrait.jpg Austin Sass:

I've lived in the area of Moscow, Idaho for most of my life. Most of my time is currently being spent in the Neurophysiological Imaging and Modeling Laboratory at the University of Idaho, where myself and team members are studying Space Flight-Associated Neuro-ocular Syndrome as it pertains to visual impairment in Astronauts. On the side, I develop flight simulation software and play music with the "Sass Brothers" band.

Computer Science
Forrest bio.jpg Seth Forrest:

I am a Technophile, I love exploring the application of technology in our daily lives. I am studying Mechanical Engineering and Computer Science in order to further investigate how tech can be integrated into our culture. I have a particular passion for the the union of the cyber and the physical, and how technology can be used to improve people's lives.

Computer Science/Mechanical Engineering

Document Archive[edit | edit source]