Skip to main content
SpringerLink
Log in

Unsupervised consumer intention and sentiment mining from microblogging data as a business intelligence tool

  • Original Paper
  • Published:
Operational Research Aims and scope Submit manuscript

Abstract

The present study aims to create a framework that analyses user posts related to a product of interest on social networking platforms. More precisely, by applying information mining techniques, posts are categorised according to the intention they express, the sentiment polarisation, and the type of opinion. The model operates based on linguistic rules, machine learning, and combinations. Six different methodologies are implemented to extract intent, sentiment, and type of opinion from a tweet. The final model automatically detects intention to buy or not to buy the product, intention to compare the product with other competitors, and finally, intention to search for information about the product. It then categorises the text according to the sentiment and depending on their expressed opinion. The dataset comprises tweets for each day of the iPhone 5’s life cycle, corresponding to 365 days. Additionally, it demonstrated that the business’s external or internal decisions affect the public purchasing audience’s opinions, sentiments, and intentions expressed on social media. Lastly, as a Business Intelligence tool, the framework recognises and analyses these points, which contribute substantially to the company’s decision-making through the findings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

Notes

  1. https://archive.org/.

  2. https://stanfordnlp.github.io/CoreNLP/.

  3. https://www.nltk.org/.

  4. https://textblob.readthedocs.io/en/dev/.

References

  • Alfonseca E, Filippova K, Delort J, et al. (2012) Pattern learning for relation extraction with a hierarchical topic model. In: The 50th annual meeting of the association for computational linguistics, proceedings of the conference, July 8–14, 2012, Jeju Island, Korea-Vol 2: Short Papers. The Association for Computer Linguistics, pp 54–59. https://aclanthology.org/P12-2011/

  • Bagozzi RP (2010) Consumer intentions. Wiley, New York. https://doi.org/10.1002/9781444316568.wiem03057

    Book  Google Scholar 

  • Becker W, Schmid O (2020) The right digital strategy for your business: an empirical analysis of the design and implementation of digital strategies in smes and lses. Bus Res 13(3):985–1005

    Article  Google Scholar 

  • Blythe M, Cairns PA (2009) Critical methods and user generated content: the iphone on youtube. In: Jr. DRO, Arthur RB, Hinckley K, et al (eds) Proceedings of the 27th international conference on human factors in computing systems, CHI 2009, Boston, MA, USA, April 4–9, 2009. ACM, pp 1467–1476, https://doi.org/10.1145/1518701.1518923

  • Carlos CS, Yalamanchi M (2012) Intention analysis for sales, marketing and customer service. In: Kay M, Boitet C (eds) COLING 2012, 24th international conference on computational linguistics, proceedings of the conference: demonstration papers, 8–15 December 2012, Mumbai, India. Indian Institute of Technology Bombay, pp 33–40. https://www.aclweb.org/anthology/C12-3005/

  • Chatterjee S (2019) Explaining customer ratings and recommendations by combining qualitative and quantitative user generated contents. Decis Support Syst 119:14–22. https://doi.org/10.1016/j.dss.2019.02.008

    Article  Google Scholar 

  • Chatterjee S, Krystyanczuk M (2017) Python social media analytics. Packt Publishing Ltd, London

    Google Scholar 

  • Chen Z, Liu B, Hsu M, et al (2013) Identifying intention posts in discussion forums. In: Vanderwende L, III HD, Kirchhoff K (eds) Human language technologies: conference of the North American chapter of the association of computational linguistics, proceedings, June 9–14, 2013, Westin Peachtree Plaza Hotel, Atlanta, Georgia, USA. The Association for Computational Linguistics, pp 1041–1050. https://www.aclweb.org/anthology/N13-1124/

  • Choi J, Yoon J, Chung J et al (2020) Social media analytics and business intelligence research: a systematic review. Inf Process Manag 57(6):102279. https://doi.org/10.1016/j.ipm.2020.102279

    Article  Google Scholar 

  • Ding X, Liu T, Duan J, et al (2015) Mining user consumption intention from social media using domain adaptive convolutional neural network. In: Bonet B, Koenig S (eds) Proceedings of the twenty-ninth AAAI conference on artificial intelligence, January 25–30, 2015, Austin, Texas, USA. AAAI Press, pp 2389–2395. http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9748

  • Eckerson WW (2010) Performance dashboards: measuring, monitoring, and managing your business. Wiley, London

    Google Scholar 

  • Faulds DJ, Mangold WG, Raju P et al (2018) The mobile shopping revolution: redefining the consumer decision process. Bus Horiz 61(2):323–338. https://doi.org/10.1016/j.bushor.2017.11.012

    Article  Google Scholar 

  • Felt M (2016) Social media and the social sciences: How researchers employ big data analytics. Big Data Soc 3(1):2053951716645,828. https://doi.org/10.1177/2053951716645828

    Article  Google Scholar 

  • Gao W, Sebastiani F (2016) From classification to quantification in tweet sentiment analysis. Soc Netw Anal Min 6(1):1–22

    Article  Google Scholar 

  • Giachanou A, Crestani F (2016) Like it or not: a survey of twitter sentiment analysis methods. ACM Comput Surv 49(2):28:1-28:41. https://doi.org/10.1145/2938640

    Article  Google Scholar 

  • Gupta V, Varshney D, Jhamtani H, et al (2014) Identifying purchase intent from social posts. In: Adar E, Resnick P, Choudhury MD, et al (eds) Proceedings of the eighth international conference on weblogs and social media, ICWSM 2014, Ann Arbor, Michigan, USA, June 1–4, 2014. The AAAI Press. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM14/paper/view/8037

  • Guzman E, Maalej W (2014) How do users like this feature? A fine grained sentiment analysis of app reviews. In: Gorschek T, Lutz RR (eds) IEEE 22nd international requirements engineering conference, RE 2014, Karlskrona, Sweden, August 25–29, 2014. IEEE Computer Society, pp 153–162, 10.1109/RE.2014.6912257,

  • Hamroun M, Gouider MS, Said LB (2016) Large scale microblogging intentions analysis with pattern based approach. In: Howlett RJ, Jain LC, Gabrys B, et al (eds) Knowledge-based and intelligent information and engineering systems: proceedings of the 20th international conference KES-2016, York, UK, 5–7 September 2016, Procedia computer science, vol 96. Elsevier, pp 1249–1257. https://doi.org/10.1016/j.procs.2016.08.169

  • He W, Wu H, Yan G et al (2015) A novel social media competitive analytics framework with sentiment benchmarks. Inf Manag 52(7):801–812. https://doi.org/10.1016/j.im.2015.04.006

    Article  Google Scholar 

  • Hollerit B, Kröll M, Strohmaier M (2013) Towards linking buyers and sellers: detecting commercial intent on twitter. In: Carr L, Laender AHF, Lóscio BF, et al (eds) 22nd international world wide web conference, WWW ’13, Rio de Janeiro, Brazil, May 13–17, 2013, Companion Volume. International World Wide Web Conferences Steering Committee / ACM, pp 629–632. https://doi.org/10.1145/2487788.2488009

  • Hung C, Lin H (2013) Using objective words in sentiwordnet to improve word-of-mouth sentiment classification. IEEE Intell Syst 28(2):47–54. https://doi.org/10.1109/MIS.2013.1

    Article  Google Scholar 

  • Hutto CJ, Gilbert E (2014) VADER: a parsimonious rule-based model for sentiment analysis of social media text. In: Adar E, Resnick P, Choudhury MD, et al (eds) Proceedings of the eighth international conference on weblogs and social media, ICWSM 2014, Ann Arbor, Michigan, USA, June 1–4, 2014. The AAAI Press. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM14/paper/view/8109

  • Jhamtani H, Chhaya N, Karwa S, et al (2015) Identifying suggestions for improvement of product features from online product reviews. In: Liu T, Scollon CN, Zhu W (eds) Social informatics-7th international conference, SocInfo 2015, Beijing, China, December 9–12, 2015, Proceedings, Lecture Notes in Computer Science, vol 9471. Springer, pp 112–119. https://doi.org/10.1007/978-3-319-27433-1_8

  • Jindal N, Liu B (2006) Identifying comparative sentences in text documents. In: Efthimiadis EN, Dumais ST, Hawking D, et al (eds) SIGIR 2006: Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval, Seattle, Washington, USA, August 6–11, 2006. ACM, pp 244–251. https://doi.org/10.1145/1148170.1148215

  • Kalamatianos G, Symeonidis S, Mallis D et al (2018) Towards the creation of an emotion lexicon for microblogging. J Syst Inf Technol 20(2):130–151. https://doi.org/10.1108/JSIT-06-2017-0040

    Article  Google Scholar 

  • Kim Y, Dwivedi R, Zhang J et al (2016) Competitive intelligence in social media twitter: iphone 6 vs. galaxy S5. Online Inf Rev 40(1):42–61. https://doi.org/10.1108/OIR-03-2015-0068

    Article  Google Scholar 

  • Kumar N, Nagalla R, Marwah T, et al (2019) Sentiment dynamics in social media news channels. CoRR abs/1908.08147. arXiv:1908.08147

  • Kurnia PF, Suharjito (2018) Business intelligence model to analyze social media information. Procedia Comput Sci 135:5–14. https://doi.org/10.1016/j.procs.2018.08.144 (The 3rd International Conference on Computer Science and Computational Intelligence (ICCSCI 2018) : Empowering Smart Technology in Digital Era for a Bette Life)

    Article  Google Scholar 

  • Ladhari R, Michaud M (2015) ewom effects on hotel booking intentions, attitudes, trust, and website perceptions. Int J Hosp Manag 46:36–45. https://doi.org/10.1016/j.ijhm.2015.01.010

    Article  Google Scholar 

  • Li N, Wu DD (2010) Using text mining and sentiment analysis for online forums hotspot detection and forecast. Decis Support Syst 48(2):354–368. https://doi.org/10.1016/j.dss.2009.09.003

    Article  Google Scholar 

  • Liu B (2010) Sentiment analysis and subjectivity. In: Indurkhya N, Damerau FJ (eds) Handbook of natural language processing, 2nd edn. Chapman and Hall/CRC, New York, pp 627–666. https://doi.org/10.1201/9781420085938-c26

    Chapter  Google Scholar 

  • Liu X, Burns AC, Hou Y (2017) An investigation of brand-related user-generated content on twitter. J Advert 46(2):236–247. https://doi.org/10.1080/00913367.2017.1297273

    Article  Google Scholar 

  • Liu Y, Bi J, Fan Z (2017) Ranking products through online reviews: a method based on sentiment analysis technique and intuitionistic fuzzy set theory. Inf Fusion 36:149–161. https://doi.org/10.1016/j.inffus.2016.11.012

    Article  Google Scholar 

  • Lumezanu C, Feamster N (2012) Observing common spam in twitter and email. In: Proceedings of the 2012 internet measurement conference. Association for Computing Machinery, New York, NY, USA, IMC ’12, pp 461–466. https://doi.org/10.1145/2398776.2398824

  • Manning CD, Surdeanu M, Bauer J, et al (2014) The stanford corenlp natural language processing toolkit. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, ACL 2014, June 22–27, 2014, Baltimore, MD, USA, System Demonstrations. The Association for Computer Linguistics, pp 55–60, https://doi.org/10.3115/v1/p14-5010

  • McKinney W, et al (2010) Data structures for statistical computing in python. In: Proceedings of the 9th Python in Science Conference, Austin, TX, pp 51–56

  • Montoyo A, Martínez-Barco P, Balahur A (2012) Subjectivity and sentiment analysis: an overview of the current state of the area and envisaged developments. Decis Support Syst 53(4):675–679. https://doi.org/10.1016/j.dss.2012.05.022

    Article  Google Scholar 

  • Morris MR, Teevan J, Panovich K (2010) A comparison of information seeking using search engines and social networks. In: Cohen WW, Gosling S (eds) Proceedings of the fourth international conference on weblogs and social media, ICWSM 2010, Washington, DC, USA, May 23–26, 2010. The AAAI Press. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/view/1518

  • Nashaat M, Ghosh A, Miller J, et al (2018) Hybridization of active learning and data programming for labeling large industrial datasets. In: IEEE international conference on big data (IEEE BigData 2018), Seattle, WA, USA, December 10–13, 2018. IEEE, pp 46–55, https://doi.org/10.1109/BigData.2018.8622459

  • Negash S, Gray P (2003) Business intelligence. In: 9th Americas conference on information systems, AMCIS 2003, Tampa, FL, USA, August 4–6, 2003. Association for Information Systems, p 423, http://aisel.aisnet.org/amcis2003/423

  • Paul SA, Hong L, Chi EH (2011) Is twitter a good place for asking questions? A characterization study. In: Adamic LA, Baeza-Yates R, Counts S (eds) Proceedings of the fifth international conference on weblogs and social media, Barcelona, Catalonia, Spain, July 17–21, 2011. The AAAI Press. http://www.aaai.org/ocs/index.php/ICWSM/ICWSM11/paper/view/2813

  • Poecze F, Ebster C, Strauss C (2018) Social media metrics and sentiment analysis to evaluate the effectiveness of social media posts. In: Shakshuki EM, Yasar A (eds) The 9th international conference on ambient systems, networks and technologies (ANT 2018) / The 8th International Conference on Sustainable Energy Information Technology (SEIT 2018) / Affiliated Workshops, May 8–11, 2018, Porto, Portugal, Procedia Computer Science, vol 130. Elsevier, pp 660–666. https://doi.org/10.1016/j.procs.2018.04.117

  • Purohit H, Dong G, Shalin VL, et al (2015) Intent classification of short-text on social media. In: 2015 IEEE international conference on Smart City/SocialCom/SustainCom 2015, Chengdu, China, December 19–21, 2015. IEEE Computer Society, pp 222–228. https://doi.org/10.1109/SmartCity.2015.75

  • Ramanand J, Bhavsar K, Pedanekar N (2010) Wishful thinking-finding suggestions and ’buy’ wishes from product reviews. In: Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text. Association for Computational Linguistics, Los Angeles, CA, pp 54–61. https://www.aclweb.org/anthology/W10-0207

  • Ratner A, Bach SH, Ehrenberg HR et al (2020) Snorkel: rapid training data creation with weak supervision. VLDB J 29(2–3):709–730. https://doi.org/10.1007/s00778-019-00552-1

    Article  Google Scholar 

  • Reeves M, Deimler MS (2009) Strategies for winning in the current and post-recession environment. Strategy Leadersh 2:1158

    Google Scholar 

  • Rui H, Liu Y, Whinston AB (2013) Whose and what chatter matters? the effect of tweets on movie sales. Decis Support Syst 55(4):863–870. https://doi.org/10.1016/j.dss.2012.12.022

    Article  Google Scholar 

  • Sahoo SR, Gupta BB (2019) Hybrid approach for detection of malicious profiles in twitter. Comput Electr Eng 76:65–81. https://doi.org/10.1016/j.compeleceng.2019.03.003

    Article  Google Scholar 

  • Saif H, Fernández M, He Y, et al (2013) Evaluation datasets for twitter sentiment analysis: a survey and a new dataset, the sts-gold. In: Battaglino C, Bosco C, Cambria E, et al (eds) Proceedings of the first international workshop on emotion and sentiment in social and expressive media: approaches and perspectives from AI (ESSEM 2013) A workshop of the XIII International Conference of the Italian Association for Artificial Intelligence (AI*IA 2013), Turin, Italy, December 3, 2013, CEUR Workshop Proceedings, vol 1096. CEUR-WS.org, pp 9–21. http://ceur-ws.org/Vol-1096/paper1.pdf

  • Salehan M, Kim DJ (2016) Predicting the performance of online consumer reviews: a sentiment mining approach to big data analytics. Decis Support Syst 81:30–40. https://doi.org/10.1016/j.dss.2015.10.006

    Article  Google Scholar 

  • Sharma N (2013) Marketing strategy on different stages plc and its marketing implications on fmcg products. Int J Mark Financ Serv Manag Res 2(3):121–136

    Google Scholar 

  • Shukri SE, Yaghi RI, Aljarah I, et al (2015) Twitter sentiment analysis: A case study in the automotive industry. In: 2015 IEEE Jordan conference on applied electrical engineering and computing technologies (AEECT), pp 1–5. https://doi.org/10.1109/AEECT.2015.7360594

  • Smith AN, Fischer E, Yongjian C (2012) How does brand-related user-generated content differ across Youtube, Facebook, and Twitter? J Interact Mark 26(2):102–113. https://doi.org/10.1016/j.intmar.2012.01.002

    Article  Google Scholar 

  • Stolcke A, Ries K, Coccaro N, et al (2000) Dialogue act modeling for automatic tagging and recognition of conversational speech. CoRR cs.CL/0006023. https://arxiv.org/abs/cs/0006023

  • Sun X, Zhang C, Li G et al (2018) Detecting users’ anomalous emotion using social media for business intelligence. J Comput Sci 25:193–200. https://doi.org/10.1016/j.jocs.2017.05.029

    Article  Google Scholar 

  • Symeonidis S, Effrosynidis D, Arampatzis A (2018) A comparative evaluation of pre-processing techniques and their interactions for twitter sentiment analysis. Expert Syst Appl 110:298–310. https://doi.org/10.1016/j.eswa.2018.06.022

    Article  Google Scholar 

  • Tsou M, Zhang H, Jung C (2017) Identifying data noises, user biases, and system errors in geo-tagged twitter messages (tweets). CoRR abs/1712.02433. arXiv:1712.02433

  • Tutunea MF, Rus RV (2012) Business intelligence solutions for sme’s. Procedia Econ Finance 3:865–870. https://doi.org/10.1016/S2212-5671(12)00242-0 (International Conference Emerging Markets Queries in Finance and Business, Petru Maior University of Tîrgu-Mures, ROMANIA, October 24th - 27th, 2012)

    Article  Google Scholar 

  • Varma P, Ré C (2018) Snuba: automating weak supervision to label training data. Proc VLDB Endow 12(3):223–236. https://doi.org/10.14778/3291264.3291268

    Article  Google Scholar 

  • Vidya NA, Fanany MI, Budi I (2015) Twitter sentiment to analyze net brand reputation of mobile phone providers. Procedia Comput Sci 72:519–526. https://doi.org/10.1016/j.procs.2015.12.159 (The Third Information Systems International Conference 2015)

    Article  Google Scholar 

  • Wang L, Yan J, Lin J et al (2017) Let the users tell the truth: self-disclosure intention and self-disclosure honesty in mobile social networking. Int J Inf Manag 37(1):1428–1440. https://doi.org/10.1016/j.ijinfomgt.2016.10.006

    Article  Google Scholar 

  • Watson HJ, Wixom BH (2007) The current state of business intelligence. Computer 40(9):96–99. https://doi.org/10.1109/MC.2007.331

    Article  Google Scholar 

  • Wiecek-Janka E, Papierz M, Kornecka M, et al (2017) Apple products: A discussion of the product life cycle. In: 4th international conference on management science and management innovation, pp 159–164

  • Wu T, Khan FM, Fisher TA, et al (2005) Posting act tagging using transformation-based learning. In: Lin TY, Ohsuga S, Liau C, et al (eds) Foundations of data mining and knowledge discovery, studies in computational intelligence, vol 6. Springer, pp 319–331. https://doi.org/10.1007/11498186_18

  • Wyrwoll C (2014) Social media-fundamentals, models, and ranking of user-generated content. Springer, Berlin. https://doi.org/10.1007/978-3-658-06984-1

    Book  Google Scholar 

  • Xu K, Liao SS, Li J et al (2011) Mining comparative opinions from customer reviews for competitive intelligence. Decis Support Syst 50(4):743–754. https://doi.org/10.1016/j.dss.2010.08.021

    Article  Google Scholar 

  • Xu X, Wang X, Li Y et al (2017) Business intelligence in online customer textual reviews: understanding consumer perceptions and influential factors. Int J Inf Manag 37(6):673–683. https://doi.org/10.1016/j.ijinfomgt.2017.06.004

    Article  Google Scholar 

  • Yao Y, Sun A (2016) Mobile phone name extraction from internet forums: a semi-supervised approach. World Wide Web 19(5):783–805. https://doi.org/10.1007/s11280-015-0361-1

    Article  Google Scholar 

  • Zhao Z, Mei Q (2013) Questions about questions: an empirical analysis of information needs on twitter. In: Schwabe D, Almeida VAF, Glaser H, et al (eds) 22nd international world wide web conference, WWW ’13, Rio de Janeiro, Brazil, May 13-17, 2013. International World Wide Web Conferences Steering Committee / ACM, pp 1545–1556. https://doi.org/10.1145/2488388.2488523

  • Zheng ZE, Fader PS, Padmanabhan B (2012) From business intelligence to competitive intelligence: inferring competitive measures using augmented site-centric data. Inf Syst Res 23(3–1):698–720. https://doi.org/10.1287/isre.1110.0385

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Database and Information Retrieval research unit, Department of Electrical and Computer Engineering, Democritus University of Thrace, Xanthi, 67100, Greece

    Symeon Symeonidis & Avi Arampatzis

  2. Information and Knowledge Representation, Retrieval, and Reasoning (IKR3) Lab, Department of Informatics, Systems, and Communication (DISCo), University of Milano-Bicocca, Milan, Italy

    Georgios Peikos

Authors
  1. Symeon Symeonidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georgios Peikos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Avi Arampatzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Symeon Symeonidis.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Symeonidis, S., Peikos, G. & Arampatzis, A. Unsupervised consumer intention and sentiment mining from microblogging data as a business intelligence tool. Oper Res Int J 22, 6007–6036 (2022). https://doi.org/10.1007/s12351-022-00714-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12351-022-00714-0

Keywords

Search

Navigation