Abstract
Causal models promise many benefits if applied correctly to machine learning tasks. However, in order to leverage fine grained causal information, it is often useful to reduce the complexity of the causal connections by producing an abstracted version of the graph. In this paper, we introduce semantic causal abstractions, a scheme for constructing abstracted causal graphs in order to provide a domain-independent approximation to formal causal abstraction for unstructured textual data. We then analyze the effects that this type of abstraction has on the performance of a causal graph-based prediction model under multiple semantic representations. Our experiments on two stock prediction tasks provide evidence for the efficacy of semantic causal abstraction to improve prediction performance and give insight into the consistency of the optimal semantic causal abstraction levels across tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
This dataset is publically available and can be accessed at https://www.kaggle.com/miguelaenlle/massive-stock-news-analysis-db-for-nlpbacktests.
- 2.
This dataset is restricted for use to the participating members of the competition only.
- 3.
This data is no longer publicly available.
References
Beckers, S., Halpern, J.Y.: Abstracting causal models. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 2678–2685 (2019)
Cer, D., et al.: Universal sentence encoder. arXiv preprint arXiv:1803.11175 (2018)
Das, S.R., Chen, M.Y.: Yahoo! for Amazon: sentiment extraction from small talk on the web. Manag. Sci. 53(9), 1375–1388 (2007)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Ding, X., Zhang, Y., Liu, T., Duan, J.: Using structured events to predict stock price movement: An empirical investigation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1415–1425 (2014)
Ester, M., et al.: A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD, vol. 96, pp. 226–231 (1996)
Harradon, M., Druce, J., Ruttenberg, B.: Causal learning and explanation of deep neural networks via autoencoded activations. arXiv preprint arXiv:1802.00541 (2018)
Hoel, E.P., Albantakis, L., Tononi, G.: Quantifying causal emergence shows that macro can beat micro. Proc. Natl. Acad. Sci. 110(49), 19790–19795 (2013)
Jammalamadaka, A., Warmsley, D., Lu, T.C.: Causal abstraction via emergence for predicting bilateral trade. In: Ma, S., Kummerfeld, E. (eds.) Proceedings of The 2021 Causal Analysis Workshop Series. Proceedings of Machine Learning Research, vol. 160, pp. 39–51. PMLR, 16 July 2021
Kao, P.W., Chen, C.C., Huang, H.H., Chen, H.H.: Ntunlpl at fincausal 2020, task 2: Improving causality detection using Viterbi decoder. In: Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation, pp. 69–73 (2020)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Liu, C., et al.: Learning causal semantic representation for out-of-distribution prediction. arXiv preprint arXiv:2011.01681 (2020)
Madumal, P., Miller, T., Sonenberg, L., Vetere, F.: Explainable reinforcement learning through a causal lens. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 2493–2500 (2020)
Magliacane, S., van Ommen, T., Claassen, T., Bongers, S., Versteeg, P., Mooij, J.M.: Domain adaptation by using causal inference to predict invariant conditional distributions. arXiv preprint arXiv:1707.06422 (2017)
Mariko, D., Abi Akl, H., Labidurie, E., Durfort, S., de Mazancourt, H., El-Haj, M.: The Financial Document Causality Detection Shared Task (FinCausal 2020). In: The 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation (FNP-FNS 2020), Barcelona, Spain (2020)
Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Mittal, A., Goel, A.: Stock prediction using twitter sentiment analysis. Standford University, CS229 (2011). http://cs229.stanford.edu/proj2011/GoelMittal-StockMarketPredictionUsingTwitterSentimentAnalysis.pdf, 15 (2012)
Perone, C.S., Silveira, R., Paula, T.S.: Evaluation of sentence embeddings in downstream and linguistic probing tasks. arXiv preprint arXiv:1806.06259 (2018)
Saedi, C., Branco, A., Rodrigues, J., Silva, J.: Wordnet embeddings. In: Proceedings of the third workshop on representation learning for NLP, pp. 122–131 (2018)
Schölkopf, B., et al.: Toward causal representation learning. Proc. IEEE 109(5), 612–634 (2021)
Schuler, K.K.: VerbNet: A broad-coverage, comprehensive verb lexicon. University of Pennsylvania (2005)
Skuza, M., Romanowski, A.: Sentiment analysis of twitter data within big data distributed environment for stock prediction. In: 2015 Federated Conference on Computer Science and Information Systems (FedCSIS), pp. 1349–1354. IEEE (2015)
Strelnikoff, S., Jammalamadaka, A., Warmsley, D.: Causal maps for multi-document summarization. In: 2020 IEEE International Conference on Big Data (Big Data), pp. 4437–4445. IEEE (2020)
Yang, C.H.H., Hung, I., Danny, T., Ouyang, Y., Chen, P.Y.: Causal inference q-network: Toward resilient reinforcement learning. arXiv preprint arXiv:2102.09677 (2021)
Zhang, C., Zhang, K., Li, Y.: A causal view on robustness of neural networks. arXiv preprint arXiv:2005.01095 (2020)
Zhang, K., Schölkopf, B., Muandet, K., Wang, Z.: Domain adaptation under target and conditional shift. In: International Conference on Machine Learning, pp. 819–827. PMLR (2013)
Zhao, S., et al.: Constructing and embedding abstract event causality networks from text snippets. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 335–344 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 IFIP International Federation for Information Processing
About this paper
Cite this paper
Strelnikoff, S., Jammalamadaka, A., Lu, TC. (2022). Semantic Causal Abstraction for Event Prediction. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds) Machine Learning and Knowledge Extraction. CD-MAKE 2022. Lecture Notes in Computer Science, vol 13480. Springer, Cham. https://doi.org/10.1007/978-3-031-14463-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-14463-9_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14462-2
Online ISBN: 978-3-031-14463-9
eBook Packages: Computer ScienceComputer Science (R0)
