Personality Analysis Using Machine Learning
|Team Name||Digital Profilers|
|Duration||Fall 2017 - Spring 2018|
The goal of this project is to develop a software capable of performing sentiment analysis on bodies of text with a given author. This tool will mimic IBM Watson's Personality Insights Service - a popular online resource and application programming interface (API) for personality classification.
There are many sentiment analysis tools available for public use, however, many have significant drawbacks, are not well documented, or perform classification with relatively poor accuracy. We plan to produce the initial framework needed to create a new sentiment analysis tool that remedies many of these problems. Specifically, we will focus on web mining Twitter tweets, storing tweets in a database with relevant statistical information, and training a machine from the collected data. In time, the results of this project may develop into a mobile app.
|Data/Web Mining Requirements|
|1||Text Sample||1200 - 3000 words|
|2||Quality of Text||Raw/Unprocessed|
|4||Retain Information||Copyable to Repository|
|6||Location Retention (i.e., the URL)||Scraper Pulls URL|
|1||Text Storage||1,000,000's of entries|
|2||Links to local copy||Cataloged|
|3||Author of text||Cataloged|
|4||IBM Watson Output||Numerics/Classifications indexed|
|Trained Machine Requirements|
|1||Process entries||1,000,000's to train|
|2||Approximate IBM Watson Output||Mean Average Error|
- Twitter mining tool capable of collecting and combining numerous tweets from verified users/authors into text samples
- Text samples passed through IBM Watson Personality Insights Service
- Database with large collection of authors, indexed IBM Watson output, and statistics
- Machine trained on collected data
For this project we are using a MySQL database. Each entry for the database contains the author’s name, the original text file, the URL of the source, the JSON objects from our scrapper (Tweepy), and the JSON objects from IBM Watson’s output. Eventually we will add the Five Factor Model information results from our machine. Until then we will collect data in order to use it to train our machine.
Sample is based on Twitter user profiles. A Python package called Tweepywas used to extract data from Twitter. Twitter restricts the number of API calls that can be made, so the scraper is be set up to run without intervention. Tweepy API information can be found HERE
Input text will be processed using an algorithm called GloVe*. GloVe uses a strategy in which relationships between words can be found by analyzing their co-occurrence. Specifically, GloVe optimizes a global log-bilinear language model with a weighted least squares objective such that the difference between the dot product and co-occurrence probability between any two word feature vectors is zero. The resulting trained word vectors can be used as features in various machine learning algorithms.
Two machine learning (ML) approaches, Gaussian Process (GP) and Convolution Neural Network (CNN), were considered. Comparisons were made between the two to determine which approach would best suit the needs of the project. Due to the statistical complexities of GP , it was decided a CNN would be implemented.
|Convolution Neural Network||
A CNN personality classifier based on Majumder et al.  implemented by SenticNet was chosen. The package includes text pre-processing, which can take the place of GloVe. Test will have to be conducted to determine whether GloVe provides any meaningful benefit.
The CNN was trained on Watson's output, which includes both percentile and raw scores for each personality trait. Mean absolute error was used as a metric to measure how far off our machine was from Watson. Below are the MAE values for each trait based on percentile and raw scores and data set sizes of 1000 and 6000 text samples. For comparison, IBM Watson achieved a MAE score of 0.12 averaged across all five traits.
 Adedoyin-Olowe, Mariam, et al. “A Survey of Data Mining Techniques for Social Media Analysis.” 2013, Journal of Data Mining \& Digital Humanities, 2014 (June 24, 2014) jdmdh:18.
 Arnoux, Pierre-Hadrien, et al. “25 Tweets to Know You: A New Model to Predict Personality with Social Media.” 2017. https://arxiv.org/ftp/arxiv/papers/1704/1704.05513.pdf
 Digman, John M. "Personality structure: Emergence of the five-factor model. Annual Review of Psychology" 1990. 41.1: 417–440.
 Kalghatgi, M. P., et al. “A Neural Network Approach to Personality Prediction based on the Big-Five Model.” International Journal of Innovative Research in Advanced Engineering, vol. 2, no. 8, 2015, pp. 56–63.
 Kaur, Arvinder, and Deepti Chopra. “Comparison of Text Mining Tools.” Institute of Electrical and Electronics Engineers, 7 Sept. 2016, pp. 186–192. ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=7784950.
 Majumder, Navonil, et al. “Deep Learning-Based Document Modeling for Personality Detection from Text.” IEEE Intelligent Systems, vol. 32, no. 2, 2017, pp. 74–79., doi:10.1109/mis.2017.23.
 Mnih, Andriy, Hinton, Geoffrey. "Three New Graphical Models for Statistical Language Modelling." 24th International Conference on Machine Learning, 2007
 Pennington, Jeffrey, Socher, Richard, Manning D., Christoper. "GloVe: Global Vectors for Word Representation." 2014. https://nlp.stanford.edu/pubs/glove.pdf.
 Rasmussen E, Carl, Williams K. I., Christoper. "Gaussian Processes for Machine Learning." 2006. http://www.gaussianprocess.org/gpml/.