Twitter Trends Analysis Using Structural Topic Modelling

Abstract

Social Networking Sites (SNS) such as Facebook and Twitter have become indispensable for netizens all over the world. They are an important source of information and entertainment for many users. Everyday increasing amounts of data is generated on these sites. This data is mostly comprised of unstructured text data(Talib, Hanif, Ayesha, & Fatima, 2016). Extracting useful information from this data would be tedious and time consuming. Humans are also error prone and can be affected by biases while computers are only influenced by the data. Computer assisted text analysis can help humans analyze this data much faster by automating the process (Talib et al., 2016). This includes techniques such as calculating the word frequency, sentiment analysis, text classification and topic modelling. This study will implement topic modelling to extract useful topics from Twitter data. Topic modelling helps us understand what a certain text corpus is talking about. It does this by structuring and organizing the data according to word co-occurrence in different documents and grouping the words into different topics. The output of the model helps us understand the most probable topics for a particular text and can be used to classify similar but previously unseen text. This study will explore how to obtain data from Twitter application programming interface (API) and the various natural language processing (NLP) techniques that will be used to prepare the text for analysis. This study will also explore the various text modelling algorithms and determine the most appropriate one for our data. Finally topics will be estimated from the text and use various visualizations to understand and evaluate the topics.