Discovering public sentiment in social media for predicting stock movement of publicly listed companies

Bing Li, Keith Chan, Carol Ou, Ruifeng Sun

Research output: Contribution to journalArticleScientificpeer-review

Abstract

The popularity of many social media sites has prompted both academic and practical research on the possibility of mining social media data for the analysis of public sentiment. Studies have suggested that public emotions shown through Twitter could be well correlated with the Dow Jones Industrial Average. However, it remains unclear how public sentiment, as reflected on social media, can be used to predict stock price movement of a particular publicly-listed company. In this study, we attempt to fill this research void by proposing a technique, called SMeDA-SA, to mine Twitter data for sentiment analysis and then predict the stock movement of specific listed companies. For the purpose of experimentation, we collected 200 million tweets that mentioned one or more of 30 companies that were listed in NASDAQ or the New York Stock Exchange. SMeDA-SA performs its task by first extracting ambiguous textual messages from these tweets to create a list of words that reflects public sentiment. SMeDA-SA then made use of a data mining algorithm to expand the word list by adding emotional phrases so as to better classify sentiments in the tweets. With SMeDA-SA, we discover that the stock movement of many companies can be predicted rather accurately with an average accuracy over 70%. This paper describes how SMeDA-SA can be used to mine social media date for sentiments. It also presents the key implications of our study.
Original languageEnglish
Pages (from-to)81-92
JournalInformation Systems
Volume69
DOIs
Publication statusPublished - Sep 2017

Fingerprint

Industry
Data mining

Keywords

  • social media analysis
  • Twitter
  • stock prediction
  • data mining
  • sentiment analysis
  • big data
  • SMeDA-SA
  • Parallel architecture

Cite this

@article{0438c9e8ce6f49f59ae72f534afda66d,
title = "Discovering public sentiment in social media for predicting stock movement of publicly listed companies",
abstract = "The popularity of many social media sites has prompted both academic and practical research on the possibility of mining social media data for the analysis of public sentiment. Studies have suggested that public emotions shown through Twitter could be well correlated with the Dow Jones Industrial Average. However, it remains unclear how public sentiment, as reflected on social media, can be used to predict stock price movement of a particular publicly-listed company. In this study, we attempt to fill this research void by proposing a technique, called SMeDA-SA, to mine Twitter data for sentiment analysis and then predict the stock movement of specific listed companies. For the purpose of experimentation, we collected 200 million tweets that mentioned one or more of 30 companies that were listed in NASDAQ or the New York Stock Exchange. SMeDA-SA performs its task by first extracting ambiguous textual messages from these tweets to create a list of words that reflects public sentiment. SMeDA-SA then made use of a data mining algorithm to expand the word list by adding emotional phrases so as to better classify sentiments in the tweets. With SMeDA-SA, we discover that the stock movement of many companies can be predicted rather accurately with an average accuracy over 70{\%}. This paper describes how SMeDA-SA can be used to mine social media date for sentiments. It also presents the key implications of our study.",
keywords = "social media analysis, Twitter, stock prediction, data mining, sentiment analysis, big data, SMeDA-SA, Parallel architecture",
author = "Bing Li and Keith Chan and Carol Ou and Ruifeng Sun",
year = "2017",
month = "9",
doi = "10.1016/j.is.2016.10.001",
language = "English",
volume = "69",
pages = "81--92",
journal = "Information Systems",
issn = "0306-4379",
publisher = "PERGAMON-ELSEVIER SCIENCE LTD",

}

Discovering public sentiment in social media for predicting stock movement of publicly listed companies. / Li, Bing; Chan, Keith; Ou, Carol; Sun, Ruifeng.

In: Information Systems, Vol. 69, 09.2017, p. 81-92.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Discovering public sentiment in social media for predicting stock movement of publicly listed companies

AU - Li, Bing

AU - Chan, Keith

AU - Ou, Carol

AU - Sun, Ruifeng

PY - 2017/9

Y1 - 2017/9

N2 - The popularity of many social media sites has prompted both academic and practical research on the possibility of mining social media data for the analysis of public sentiment. Studies have suggested that public emotions shown through Twitter could be well correlated with the Dow Jones Industrial Average. However, it remains unclear how public sentiment, as reflected on social media, can be used to predict stock price movement of a particular publicly-listed company. In this study, we attempt to fill this research void by proposing a technique, called SMeDA-SA, to mine Twitter data for sentiment analysis and then predict the stock movement of specific listed companies. For the purpose of experimentation, we collected 200 million tweets that mentioned one or more of 30 companies that were listed in NASDAQ or the New York Stock Exchange. SMeDA-SA performs its task by first extracting ambiguous textual messages from these tweets to create a list of words that reflects public sentiment. SMeDA-SA then made use of a data mining algorithm to expand the word list by adding emotional phrases so as to better classify sentiments in the tweets. With SMeDA-SA, we discover that the stock movement of many companies can be predicted rather accurately with an average accuracy over 70%. This paper describes how SMeDA-SA can be used to mine social media date for sentiments. It also presents the key implications of our study.

AB - The popularity of many social media sites has prompted both academic and practical research on the possibility of mining social media data for the analysis of public sentiment. Studies have suggested that public emotions shown through Twitter could be well correlated with the Dow Jones Industrial Average. However, it remains unclear how public sentiment, as reflected on social media, can be used to predict stock price movement of a particular publicly-listed company. In this study, we attempt to fill this research void by proposing a technique, called SMeDA-SA, to mine Twitter data for sentiment analysis and then predict the stock movement of specific listed companies. For the purpose of experimentation, we collected 200 million tweets that mentioned one or more of 30 companies that were listed in NASDAQ or the New York Stock Exchange. SMeDA-SA performs its task by first extracting ambiguous textual messages from these tweets to create a list of words that reflects public sentiment. SMeDA-SA then made use of a data mining algorithm to expand the word list by adding emotional phrases so as to better classify sentiments in the tweets. With SMeDA-SA, we discover that the stock movement of many companies can be predicted rather accurately with an average accuracy over 70%. This paper describes how SMeDA-SA can be used to mine social media date for sentiments. It also presents the key implications of our study.

KW - social media analysis

KW - Twitter

KW - stock prediction

KW - data mining

KW - sentiment analysis

KW - big data

KW - SMeDA-SA

KW - Parallel architecture

U2 - 10.1016/j.is.2016.10.001

DO - 10.1016/j.is.2016.10.001

M3 - Article

VL - 69

SP - 81

EP - 92

JO - Information Systems

JF - Information Systems

SN - 0306-4379

ER -