Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Human–AI adaptive dynamics drives the emergence of information cocoons

Nature Machine Intelligence volume 5, pages 1214–1224 (2023)Cite this article

Subjects

Abstract

Despite AI-driven recommendation algorithms being widely adopted to counter information overload, substantial evidence suggests that they are building cocoons of homogeneous contents and viewpoints, further aggravating social polarization and prejudice. Curbing these perils requires a deep insight into the origin of information cocoons. Here we investigate information cocoons in the real world using two large datasets and find that a large number of users are trapped in information cocoons. Further empirical analysis suggests that two ingredients, each corresponding to a fundamental mechanism in human–AI interaction systems, are correlated with the loss of information diversity. Grounded on the empirical findings, we derive a mechanistic model for the adaptive information dynamics in complex human–AI interaction systems governed by these fundamental mechanisms. It allows us to predict critical transitions between three states: diversification, partial information cocoons, and deep information cocoons. Our work not only empirically traces real-world information cocoons in two representative scenarios, but also theoretically unearths basic mechanisms governing the emergence of information cocoons. We provide a theoretical method for understanding major social issues resulting from adaptive information dynamics in complex human–AI interaction systems.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Empirical observations on ICs and our proposed adaptive information dynamics model.
Fig. 2: Effects of β and ∣γ+∣ on ICs.
Fig. 3: Effects of ∣γ−∣ and σ on ICs.
Fig. 4: State diagram.

Similar content being viewed by others

Data availability

The news dataset5 is available at https://msnews.github.io/. For commercial reasons, we anonymize the specific name of the video platform. We present the video dataset at https://github.com/tsinghua-fib-lab/Adaptive-Information-Dynamic-Model (refs. 39,40). In the GitHub repository, we provide the behavioural data aggregated to individual granularity and the processed data for Figs. 1–4. Source data are provided with this paper.

Code availability

The code used in this research is available at https://github.com/tsinghua-fib-lab/Adaptive-Information-Dynamic-Model (refs. 39,40).

References

  1. Tagliabue, J. et al. A challenge for rounded evaluation of recommender systems. Nat. Mach. Intell. 5, 181–182 (2023).

  2. Ricci, F., Rokach, L. & Shapira, B. Recommender Systems Handbook (Springer, 2022).

  3. Zhang, S., Yao, L., Sun, A. & Tay, Y. Deep learning based recommender system: a survey and new perspectives. ACM Comput. Surv. 52, 1–38 (2019).

    Article  Google Scholar 

  4. Bakshy, E., Messing, S. & Adamic, L. A. Exposure to ideologically diverse news and opinion on Facebook. Science 348, 1130–1132 (2015).

    Article  MathSciNet  MATH  Google Scholar 

  5. Wu, F. et al. Mind: a large-scale dataset for news recommendation. In Proc. 58th Annual Meeting of the Association for Computational Linguistics (eds Jurafsky, D. et al.) 3597–3606 (Association for Computational Linguistics, 2020).

  6. Covington, P., Adams, J. & Sargin, E. Deep neural networks for YouTube recommendations. In RecSys '16: 10th ACM Conference on Recommender Systems 191–198 (Association for Computing Machinery, 2016).

  7. Davidson, J. et al. The YouTube video recommendation system. In Proc. Fourth ACM Conference on Recommender Systems, RecSys ’10 293–296 (Association for Computing Machinery, 2010).

  8. Santos, F. P., Lelkes, Y. & Levin, S. A. Link recommendation algorithms and dynamics of polarization in online social networks. Proc. Natl Acad. Sci. USA 118, e2102141118 (2021).

    Article  MathSciNet  Google Scholar 

  9. Sunstein, C. R. Infotopia: How Many Minds Produce Knowledge (Oxford University Press, 2006).

  10. Nguyen, T. T., Hui, P.-M., Harper, F. M., Terveen, L. & Konstan, J. A. Exploring the filter bubble: the effect of using recommender systems on content diversity. In WWW '14 Companion: Proc. 23rd International Conference on World Wide Web 677–686 (Association for Computing Machinery, 2014).

  11. Chaney, A. J. B., Stewart, B. M. & Engelhardt, B. E. How algorithmic confounding in recommendation systems increases homogeneity and decreases utility. In Proc. 12th ACM Conference on Recommender Systems, RecSys ’18 224–232 (Association for Computing Machinery, 2018).

  12. Algorithmic recommendations, anyone? Nat. Mach. Intell. 5, 95 (2023).

  13. Liu, J., Huang, S., Aden, N. M., Johnson, N. F. & Song, C. Emergence of polarization in coevolving networks. Phys. Rev. Lett. 130, 037401 (2023).

    Article  MathSciNet  Google Scholar 

  14. Baumann, F., Lorenz-Spreen, P., Sokolov, I. M. & Starnini, M. Modeling echo chambers and polarization dynamics in social networks. Phys. Rev. Lett. 124, 048301 (2020).

    Article  MathSciNet  Google Scholar 

  15. Cinelli, M., De Francisci Morales, G., Galeazzi, A., Quattrociocchi, W. & Starnini, M. The echo chamber effect on social media. Proc. Natl Acad. Sci. USA 118, e2023301118 (2021).

    Article  Google Scholar 

  16. Leonard, D. & Sensiper, S. The role of tacit knowledge in group innovation. Calif. Manag. Rev. 40, 112–132 (1998).

    Article  Google Scholar 

  17. Munson, S. A. & Resnick, P. Presenting diverse political opinions: how and how much. In Proc. SIGCHI Conference on Human Factors in Computing Systems 1457–1466 (Association for Computing Machinery, 2010).

  18. Garimella, K., De Francisci Morales, G., Gionis, A. & Mathioudakis, M. Political discourse on social media: echo chambers, gatekeepers, and the price of bipartisanship. In WWW '18: Proc. 2018 World Wide Web Conference 913–922 (International World Wide Web Conferences Steering Committee, 2018).

  19. Schmidt, A. L. et al. Anatomy of news consumption on Facebook. Proc. Natl Acad. Sci. USA 114, 3035–3039 (2017).

    Article  Google Scholar 

  20. Kitchens, B., Johnson, S. L. & Gray, P. Understanding echo chambers and filter bubbles: the impact of social media on diversification and partisan shifts in news consumption. MIS Q. 44, 1619–1649 (2020).

  21. Kalimeris, D., Bhagat, S., Kalyanaraman, S. & Weinsberg, U. Preference amplification in recommender systems. In Proc. 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining 805–815 (Association for Computing Machinery, 2021).

  22. Korbel, J., Lindner, S. D., Pham, T. M., Hanel, R. & Thurner, S. Homophily-based social group formation in a spin glass self-assembly framework. Phys. Rev. Lett. 130, 057401 (2023).

    Article  MathSciNet  Google Scholar 

  23. Lorenz-Spreen, P., Oswald, L., Lewandowsky, S. & Hertwig, R. A systematic review of worldwide causal and correlational evidence on digital media and democracy. Nat. Hum. Behav. 7, 74–101 (2023).

    Article  Google Scholar 

  24. Flamino, J. et al. Political polarization of news media and influencers on Twitter in the 2016 and 2020 US presidential elections. Nat. Hum. Behav. 7, 904–916 (2023).

  25. Levy, R. Social media, news consumption, and polarization: evidence from a field experiment. Am. Econ. Rev. 111, 831–870 (2021).

    Article  Google Scholar 

  26. Bail, C. A. et al. Exposure to opposing views on social media can increase political polarization. Proc. Natl Acad. Sci. USA 115, 9216–9221 (2018).

    Article  Google Scholar 

  27. Rudin, C. Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1, 206–215 (2019).

    Article  Google Scholar 

  28. Castelvecchi, D. Can we open the black box of AI?. Nature 538, 20–23 (2016).

    Article  Google Scholar 

  29. Kunaver, M. & Požrl, T. Diversity in recommender systems—a survey. Knowl.-Based Syst. 123, 154–162 (2017).

    Article  Google Scholar 

  30. Liu, P., Shivaram, K., Culotta, A., Shapiro, M. A. & Bilgic, M. The interaction between political typology and filter bubbles in news recommendation algorithms. In WWW '21: Proc. Web Conference 2021 3791–3801 (Association for Computing Machinery, 2021).

  31. Rendle, S., Freudenthaler, C., Gantner, Z. & Schmidt-Thieme, L. BPR: Bayesian personalized ranking from implicit feedback. In UAI ’09: Proc. 25th Conference on Uncertainty in Artificial Intelligence 452–461 (AUAI Press, 2009).

  32. Ding, J., Quan, Y., He, X., Li, Y. & Jin, D. Reinforced negative sampling for recommendation with exposure data. In Proc. 28th International Joint Conference on Artificial Intelligence, IJCAI-19 (ed. Kraus, S.) 2230–2236 (International Joint Conferences on Artificial Intelligence, 2019).

  33. Su, X. & Khoshgoftaar, T. M. A survey of collaborative filtering techniques. Adv. Artif. Intell. 2009, 421425 (2009).

  34. Kobayashi, M. & Takeda, K. Information retrieval on the web. ACM Comput. Surv. 32, 144–173 (2000).

    Article  Google Scholar 

  35. König, M. D., Levchenko, A., Rogers, T. & Zilibotti, F. Aggregate fluctuations in adaptive production networks. Proc. Natl Acad. Sci. USA 119, e2203730119 (2022).

    Article  Google Scholar 

  36. Itô, K. On Stochastic Differential Equations (American Mathematical Society, 1951).

  37. Clifford, P. & Sudbury, A. A model for spatial conflict. Biometrika 60, 581–588 (1973).

    Article  MathSciNet  MATH  Google Scholar 

  38. Holley, R. A. & Liggett, T. M. Ergodic theorems for weakly interacting infinite systems and the voter model. Ann. Probab. 3, 643–663 (1975).

  39. Piao, J. et al. Open code for in-review natmachintell-a23038004 “Human–AI adaptive dynamics drive emergence of information cocoons”. Code Ocean https://doi.org/10.24433/CO.6503936.v1 (2023).

  40. Piao, J. et al. tsinghua-fib-lab/Adaptive-Information-Dynamic- Model: NMI. Zenodo https://doi.org/10.5281/zenodo.8265474 (2023).

Download references

Acknowledgements

We thank J. Ding, Z. Chen and C. Song for discussions and comments on the manuscript. This work was supported in part by the National Key Research and Development Program of China under grant 2020AAA0106000 to Y.L., the National Natural Science Foundation of China under grants 72104126 to F.Z., 71721002 to J.S., U1936217 and U22B2057 to Y.L. The funders had no role in study design, data collection, data analysis, decision to publish, or preparation of the manuscript.

Author information

Author notes

  1. These authors contributed equally: Jinghua Piao, Jiazhen Liu.

Authors and Affiliations

  1. Beijing National Research Center for Information Science and Technology (BNRist), Department of Electronic Engineering, Tsinghua University, Beijing, People’s Republic of China

    Jinghua Piao, Jiazhen Liu & Yong Li

  2. School of Public Policy and Management, Tsinghua University, Beijing, People’s Republic of China

    Fang Zhang & Jun Su

Authors
  1. Jinghua Piao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiazhen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jun Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.P., J.L. and Y.L. designed the model. J.P. performed the experiments and prepared the figures. J.L. conducted the theoretical analysis. F.Z., J.S. and Y.L. provided critical revisions. All authors jointly participated in the writing of the manuscript.

Corresponding author

Correspondence to Yong Li.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks the anonymous reviewers for their contribution to the peer review of this work. Primary Handling Editor: Liesbeth Venema, in collaboration with the Nature Machine Intelligence team.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Information.

Source data

Source Data Fig. 1

Statistical source data.

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Piao, J., Liu, J., Zhang, F. et al. Human–AI adaptive dynamics drives the emergence of information cocoons. Nat Mach Intell 5, 1214–1224 (2023). https://doi.org/10.1038/s42256-023-00731-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-023-00731-4

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics