Skip to main content
Springer Nature Link
Log in

Causal Subgraphs and Information Bottlenecks: Redefining OOD Robustness in Graph Neural Networks

  • Conference paper
  • First Online:
Computer Vision – ECCV 2024 (ECCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15146))

Included in the following conference series:

  • 330 Accesses

Abstract

Graph Neural Networks (GNNs) are increasingly popular in processing graph-structured data, yet they face significant challenges when training and testing distributions diverge, common in real-world scenarios. This divergence often leads to substantial performance drops in GNN models. To address this, we introduce a novel approach that effectively enhances GNN performance in Out-of-Distribution (OOD) scenarios, called Causal Subgraphs and Information Bottlenecks (CSIB). CSIB is guided by causal modeling principles to generate causal subgraphs while concurrently considering both Fully Informative Invariant Features (FIIF) and Partially Informative Invariant Features (PIIF) situations. Our approach uniquely combines the principles of invariant risk minimization and graph information bottleneck. This integration not only guides the generation of causal subgraphs but also underscores the necessity of balancing invariant principles with information compression in the face of various distribution shifts. We validate our model through extensive experiments across diverse shift types, demonstrating its effectiveness in maintaining robust performance under OOD conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahuja, K., et al.: Invariance principle meets information bottleneck for out-of-distribution generalization. Adv. Neural. Inf. Process. Syst. 34, 3438–3450 (2021)

    Google Scholar 

  2. Arjovsky, M., Bottou, L., Gulrajani, I., Lopez-Paz, D.: Invariant risk minimization. arXiv preprint arXiv:1907.02893 (2019)

  3. Chang, S., Zhang, Y., Yu, M., Jaakkola, T.: Invariant rationalization. In: International Conference on Machine Learning, pp. 1448–1458. PMLR (2020)

    Google Scholar 

  4. Chen, Y., et al.: Learning causally invariant representations for out-of-distribution generalization on graphs. Adv. Neural. Inf. Process. Syst. 35, 22131–22148 (2022)

    Google Scholar 

  5. Ding, K., et al.: Graph convolutional networks for multi-modality medical imaging: Methods, architectures, and clinical applications. arXiv preprint arXiv:2202.08916 (2022)

  6. Fan, S., Wang, X., Mo, Y., Shi, C., Tang, J.: Debiasing graph neural networks via learning disentangled causal substructure. Adv. Neural. Inf. Process. Syst. 35, 24934–24946 (2022)

    Google Scholar 

  7. Fan, S., Wang, X., Shi, C., Cui, P., Wang, B.: Generalizing graph neural networks on out-of-distribution graphs. IEEE Trans. Pattern Anal. Mach. Intell. 46(1), 322–337 (2023)

    Google Scholar 

  8. Geirhos, R., et al.: Shortcut learning in deep neural networks. Nat. Mach. Intell. 2(11), 665–673 (2020)

    Article  Google Scholar 

  9. Gilbert, E.N.: Random graphs. Ann. Math. Stat. 30(4), 1141–1144 (1959)

    Article  Google Scholar 

  10. Gui, S., Li, X., Wang, L., Ji, S.: GOOD: a graph out-of-distribution benchmark. Adv. Neural. Inf. Process. Syst. 35, 2059–2073 (2022)

    Google Scholar 

  11. Gui, S., Liu, M., Li, X., Luo, Y., Ji, S.: Joint learning of label and environment causal independence for graph out-of-distribution generalization. In: Advances in Neural Information Processing Systems, vol. 36 (2024)

    Google Scholar 

  12. He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., Wang, M.: LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 639–648 (2020)

    Google Scholar 

  13. Huang, J., Li, R.: Adaptive graph convolutional neural network and its biomedical applications. In: State of the Art in Neural Networks and Their Applications, pp. 105–132. Elsevier (2023)

    Google Scholar 

  14. Huszar, F.: Invariant risk minimization: An information theoretic view (2019)

    Google Scholar 

  15. Ji, Y., et al.: DrugOOD: out-of-distribution (OOD) dataset curator and benchmark for AI-aided drug discovery–a focus on affinity prediction problems with noise annotations. arXiv preprint arXiv:2201.09637 (2022)

  16. Krueger, D., et al.: Out-of-distribution generalization via risk extrapolation (REx). In: International Conference on Machine Learning, pp. 5815–5826. PMLR (2021)

    Google Scholar 

  17. Li, H., Wang, X., Zhang, Z., Zhu, W.: OOD-GNN: out-of-distribution generalized graph neural network. IEEE Trans. Knowl. Data Eng. 35(7), 7328–7340 (2022)

    Google Scholar 

  18. Li, K., et al.: Reliable representations make a stronger defender: unsupervised structure refinement for robust GNN. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 925–935 (2022)

    Google Scholar 

  19. Li, K., Liu, Y., Ao, X., He, Q.: Revisiting graph adversarial attack and defense from a data distribution perspective. In: The Eleventh International Conference on Learning Representations (2022)

    Google Scholar 

  20. Liu, G., Zhao, T., Xu, J., Luo, T., Jiang, M.: Graph rationalization with environment-based augmentations. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1069–1078 (2022)

    Google Scholar 

  21. Liu, S., Wang, H., Liu, W., Lasenby, J., Guo, H., Tang, J.: Pre-training molecular graph representation with 3D geometry. arXiv preprint arXiv:2110.07728 (2021)

  22. Ma, H., An, W., Wang, Y., Sun, H., Huang, R., Huang, J.: Deep graph learning with property augmentation for predicting drug-induced liver injury. Chem. Res. Toxicol. 34(2), 495–506 (2020)

    Article  Google Scholar 

  23. Ma, H., et al.: Cross-dependent graph neural networks for molecular property prediction. Bioinformatics 38(7), 2003–2009 (2022)

    Article  Google Scholar 

  24. Ma, H., Jiang, F., Rong, Y., Guo, Y., Huang, J.: Robust self-training strategy for various molecular biology prediction tasks. In: Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, pp. 1–5 (2022)

    Google Scholar 

  25. Miao, S., Liu, M., Li, P.: Interpretable and generalizable graph learning via stochastic attention mechanism. In: International Conference on Machine Learning, pp. 15524–15543. PMLR (2022)

    Google Scholar 

  26. Pearl, J.: Causality. Cambridge University Press, Cambridge (2009)

    Google Scholar 

  27. Peters, J., Bühlmann, P., Meinshausen, N.: Causal inference by using invariant prediction: identification and confidence intervals. J. R. Stat. Soc. Ser. B Stat. Methodol. 78(5), 947–1012 (2016)

    Article  MathSciNet  Google Scholar 

  28. Rong, Y., Huang, W., Xu, T., Huang, J.: DropEdge: Towards deep graph convolutional networks on node classification. arXiv preprint arXiv:1907.10903 (2019)

  29. Sagawa, S., Koh, P.W., Hashimoto, T.B., Liang, P.: Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv preprint arXiv:1911.08731 (2019)

  30. Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2008)

    Article  Google Scholar 

  31. Shi, C., Xu, M., Zhu, Z., Zhang, W., Zhang, M., Tang, J.: GraphAF: a flow-based autoregressive model for molecular graph generation. arXiv preprint arXiv:2001.09382 (2020)

  32. Shi, Y., et al.: ChatGraph: interpretable text classification by converting ChatGPT knowledge to graphs. In: 2023 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 515–520. IEEE (2023)

    Google Scholar 

  33. Sui, Y., Wang, X., Wu, J., Lin, M., He, X., Chua, T.S.: Causal attention for interpretable and generalizable graph classification. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 1696–1705 (2022)

    Google Scholar 

  34. Wang, X., He, X., Wang, M., Feng, F., Chua, T.S.: Neural graph collaborative filtering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 165–174 (2019)

    Google Scholar 

  35. Wu, S., Sun, F., Zhang, W., Xie, X., Cui, B.: Graph neural networks in recommender systems: a survey. ACM Comput. Surv. 55(5), 1–37 (2022)

    Article  Google Scholar 

  36. Wu, T., Ren, H., Li, P., Leskovec, J.: Graph information bottleneck. Adv. Neural. Inf. Process. Syst. 33, 20437–20448 (2020)

    Google Scholar 

  37. Wu, Y.X., Wang, X., Zhang, A., He, X., Chua, T.S.: Discovering invariant rationales for graph neural networks. arXiv preprint arXiv:2201.12872 (2022)

  38. Yan, Y., He, S., Yu, Z., Yuan, J., Liu, Z., Chen, Y.: Investigation of customized medical decision algorithms utilizing graph neural networks. arXiv preprint arXiv:2405.17460 (2024)

  39. Yang, J., et al.: Hierarchical graph capsule network. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 10603–10611 (2021)

    Google Scholar 

  40. Yang, N., Zeng, K., Wu, Q., Jia, X., Yan, J.: Learning substructure invariance for out-of-distribution molecular representations. Adv. Neural. Inf. Process. Syst. 35, 12964–12978 (2022)

    Google Scholar 

  41. Yu, J., Xu, T., Rong, Y., Bian, Y., Huang, J., He, R.: Graph information bottleneck for subgraph recognition. arXiv preprint arXiv:2010.05563 (2020)

Download references

Acknowledgements

This work was partially supported by US National Science Foundation IIS-2412195, CCF-2400785 and the Cancer Prevention and Research Institute of Texas (CPRIT) award (RP230363).

Author information

Authors and Affiliations

  1. The University of Texas at Arlington, Arlington, TX, 76010, USA

    Weizhi An, Wenliang Zhong, Feng Jiang, Hehuan Ma & Junzhou Huang

Authors
  1. Weizhi An
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenliang Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hehuan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junzhou Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junzhou Huang .

Editor information

Editors and Affiliations

  1. University of Birmingham, Birmingham, UK

    Aleš Leonardis

  2. University of Trento, Trento, Italy

    Elisa Ricci

  3. Technical University of Darmstadt, Darmstadt, Germany

    Stefan Roth

  4. Princeton University, Princeton, NJ, USA

    Olga Russakovsky

  5. Czech Technical University in Prague, Prague, Czech Republic

    Torsten Sattler

  6. École des Ponts ParisTech, Marne-la-Vallée, France

    Gül Varol

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

An, W., Zhong, W., Jiang, F., Ma, H., Huang, J. (2025). Causal Subgraphs and Information Bottlenecks: Redefining OOD Robustness in Graph Neural Networks. In: Leonardis, A., Ricci, E., Roth, S., Russakovsky, O., Sattler, T., Varol, G. (eds) Computer Vision – ECCV 2024. ECCV 2024. Lecture Notes in Computer Science, vol 15146. Springer, Cham. https://doi.org/10.1007/978-3-031-73223-2_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-73223-2_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-73222-5

  • Online ISBN: 978-3-031-73223-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics