Skip to main content
SpringerLink
Log in

SISSOS: intervention of tabular data and its applications

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Causality is getting more and more attention, and its core idea is counterfactual and intervention. However, the current intervention model requires some prior knowledge, and lacks universality. The paper presents a novel solution called Search the Intervention Sample in Sparse Operation Space (SISSOS). SISSOS introduces variational inference and realizes intervention, that’s feature manipulation at the attribute level. SISSOS is for tabular data and uses sparse space to solve attribute coupling. SISSOS is applied to counterfactual and model interpretation in experiments. In the counterfactual experiment, the proposed solution was proven to find the correct causal effect without any prior knowledge. In the model interpretation experiment, a trained time series neural network with high accuracy was proved by the proposed solution to conform to prior knowledge. Compared with the previous method, the proposed method does not require prior knowledge and its intervention effect is better.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data Availability

Some raw/processed data required to reproduce these findings cannot be shared at this time as the data also forms part of an ongoing study.

Code Availability

Some code generated or used during the study are available from the corresponding author by request.

References

  1. Adler P, Falk C, Friedler SA, Nix T, Rybeck G, Scheidegger C, Smith B, Venkatasubramanian S (2018) Auditing black-box models for indirect influence. Knowl Inf Syst 54(1):95–122

    Article  Google Scholar 

  2. Bai M, Bai X, Zhang Z, Bai M, Yang B (2005) Treatment of red tide in ocean using non-thermal plasma based advanced oxidation technology. Plasma Chem Plasma Process 25(5):539–550

    Article  Google Scholar 

  3. Besserve M, Mehrjou A, Sun R, Schölkopf B (2020) Counterfactuals uncover the modular structure of deep generative models. In: Eighth international conference on learning representations (ICLR), p 2020

  4. Blei DM, Kucukelbir A, McAuliffe JD (2017) Variational inference: a review for statisticians. J Am Stat Assoc 112(518):859–877

    Article  MathSciNet  Google Scholar 

  5. Bollen KA, Pearl J (2013) Eight myths about causality and structural equation models. Handbook of causal analysis for social research. Springer, Berlin, pp 301–328

    Book  Google Scholar 

  6. David KE, Keane H, Noh JM (2019) Ganchors: realistic image perturbation distributions for anchors using generative models

  7. Feldman M, Friedler SA, Moeller J, Scheidegger C, Venkatasubramanian S (2015) Certifying and removing disparate impact. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp 259–268

  8. Goodfellow IJ, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the 27th International conference on neural information processing systems-volume, vol 2, pp 2672–2680

  9. Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D (2018) A survey of methods for explaining black box models. ACM computing surveys (CSUR) 51(5):1–42

    Article  Google Scholar 

  10. Hernán MA (2004) A definition of causal effect for epidemiological research. J Epidemiol Commun Health 58(4):265–271

    Article  Google Scholar 

  11. Hu Z, Yang Z, Salakhutdinov R, Xing EP (2018) On unifying deep generative models. In: International conference on learning representations

  12. Huang C, Qi Y (1997) The abundance cycle and influence factors on red tide phenomena of noctiluca scintillans (dinophyceae) in dapeng bay, the south China sea. J Plankton Res 19(3):303–318

    Article  Google Scholar 

  13. Kingma DP, Dhariwal P (2018) Glow: generative flow with invertible 1× 1 convolutions. In: Proceedings of the 32nd international conference on neural information processing systems, pp 10236–10245

  14. Kingma DP, Welling M (2014) Auto-encoding variational bayes. Stat 1050:1

    MATH  Google Scholar 

  15. Le T, Wang S, Lee D (2020) Grace: generating concise and informative contrastive sample to explain neural network model’s prediction

  16. Ling-jiang S (2009) Analysis on the circulation patterns and hydrometeorology of akashiwo sanguinea red tide outbreak in xiamen sea area [j]. J Fujian Fish 3

  17. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, Van Der Laak JA, Van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88

    Article  Google Scholar 

  18. Luo Y, Peng J, Ma J (2020) When causal inference meets deep learning. Nat Mach Intell 2(8):426–427

    Article  Google Scholar 

  19. Moosavi-Dezfooli SM, Fawzi A, Frossard P (2016) Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2574–2582

  20. Nair V, Hinton GE (2009) 3d object recognition with deep belief nets. Adv Neural Inf Process Syst 22:1339–1347

    Google Scholar 

  21. Neuberg LG (2003) Causality: models, reasoning, and inference. JSTOR

  22. Olshausen BA, Field DJ (1997) Sparse coding with an overcomplete basis set: A strategy employed by v1? Vis Res 37(23):3311–3325

    Article  Google Scholar 

  23. Pearl J (2018) Causal and counterfactual inference. The Handbook of Rationality, pp 1–41

  24. Pearl J, et al. (2009) Causal inference in statistics: An overview. Stat Surv 3:96–146

    Article  MathSciNet  Google Scholar 

  25. Pingault JB, O’reilly PF, Schoeler T, Ploubidis GB, Rijsdijk F, Dudbridge F (2018) Using genetic data to strengthen causal inference in observational research. Nat Rev Genet 19(9):566–580

    Article  Google Scholar 

  26. Qiao J, Pu T, Wang X (2021) Renewable scenario generation using controllable generative adversarial networks with transparent latent space. CSEE J Power Energy Syst 7(1):66–77. 10.17775/CSEEJPES.2020.00700

    Article  Google Scholar 

  27. Ribeiro MT, Singh S, Guestrin C (2016) “Why should i trust you?” explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135–1144

  28. Ribeiro MT, Singh S, Guestrin C (2018) Anchors: High-precision model-agnostic explanations. In: Proceedings of the AAAI conference on artificial intelligence, vol 32

  29. Schlegel U, Arnout H, El-Assady M, Oelke D, Keim DA (2019) Towards a rigorous evaluation of xai methods on time series. In: 2019 IEEE/CVF international conference on computer vision workshop (ICCVW), pp 4321–4325

  30. Shen Y, Gu J, Tang X, Zhou B (2020) Interpreting the latent space of gans for semantic face editing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9243–9252

  31. Su J (2018) Variational inference: A unified framework of generative models and some revelations. arXiv:180705936

  32. Tas S (2015) A prolonged red tide of heterocapsa triquetra (ehrenberg) f. stein (dinophyceae) and phytoplankton succession in a eutrophic estuary in turkey. Mediterr Mar Sci 16(3):621–627

    Article  Google Scholar 

  33. Xu C, Huang M, Du Q (2010) Ecological characteristics of important red tide species in fujian coastal waters. J Oceanogr Taiwan Strait 29(3):434–441

    Google Scholar 

  34. Yao L, Chu Z, Li S, Li Y, Gao J, Zhang A (2020) A survey on causal inference. arXiv:200202770

Download references

Funding

This work was supported by the National Key R&D Program of China [grant number 2016YFC1401900]; the Key Laboratory of Digital Ocean, SOA, China [grant number B201801030]

Author information

Authors and Affiliations

  1. Department of Computer Engineering and Science, Shanghai University, 99 Shangda Road, Shanghai, 200444, China

    Yucheng Liu, Jie Yu & Lingyu Xu

  2. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, 99 Shangda Road, Shanghai, 200444, China

    Lingyu Xu

  3. East Sea Information Center Soa China, 1168 Jingqiao Road, Shanghai, 200136, China

    Lei Wang

  4. National Marine Data Information Center, 93 Liuwei Road, Tianjin, China

    Jinkun Yang

Authors
  1. Yucheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lingyu Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jinkun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jie Yu or Lingyu Xu.

Ethics declarations

Competing interests

None of the authors have potential conflicts of interest to be disclosed.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Yu, J., Xu, L. et al. SISSOS: intervention of tabular data and its applications. Appl Intell 52, 1044–1058 (2022). https://doi.org/10.1007/s10489-021-02382-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02382-7

Keywords

Search

Navigation