Skip to main content
SpringerLink
Log in

Addressing the Unsustainability of Deep Neural Networks with Next-Gen AI

  • Conference paper
  • First Online:
Artificial General Intelligence (AGI 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13921))

Included in the following conference series:

  • 837 Accesses

Abstract

Humanity is currently facing one of its biggest challenges to date: The climate crisis. As a result, most industry sectors are reassessing their ways of working to be better equipped to address their share of the situation. The digital sector often gets set aside in such considerations in talk about the green transition because a significant amount of its work consists of optimizing processes that can save resources. Deep neural networks (DNNs) have gained great attraction and have shown good results regarding process automation. We argue that there are well-known and lesser known negative side-effects to automation frameworks based on DNNs (and related technologies) in terms of energy consumption, pollution, and social equality, that must be questioned. We analyze the operating principles and deployment methods of DNNs, the new era of automation efforts this has launched, and argue on this basis that their continued use is both unsustainable and indefensible. Using three examples of ongoing research, we explain how alternative approaches to develop more general machine intelligence are well-poised to power the next phase of AI-based automation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    According to Statista, IT-related revenue has a predicted annual growth rate of 6.86% and a predicted market volume of US$1,570.00bn by 2027 (https://www.statista.com/outlook/tmo/it-services/worldwide – accessed March 1st, 2023).

  2. 2.

    Our use of the term ‘contemporary AI’ refers to a set of methodologies that are currently in active experimentation or use in industry, including but not limited to reinforcement learning, ANNs of all kinds, and other well-known methods.

  3. 3.

    For instance, the annual prediction that “full self-driving cars will be available next year” has been updated at a rate of one year per year by Tesla’s CEO (“Watch Elon Musk Promise Self-Driving Cars ‘Next Year’ Every Year Since 2014,” https://futurism.com/video-elon-musk-promising-self-driving-cars — accessed March 1st, 2023.

  4. 4.

    The UN defines ‘sustainable development’ as harmony between economic growth, social inclusion, and environmental protection. https://www.un.org/sustainabledevelopment/development-agenda/ — accessed April 4th, 2023.

  5. 5.

    ChatGPT has passed the Wharton Exam, US medical licensing exam, law school exam, and others. (https://www.businessinsider.com/list-here-are-the-exams-chatgpt-has-passed-so-far-2023-1?r=US&IR=T#wharton-mba-exam-1 — accessed March 4th, 2023).

  6. 6.

    As of April 2023, the price is $20 a month for reliable and fast access to ChatGPT, although a free version with slower response is still available. (https://openai.com/blog/chatgpt-plus — accessed April 4th, 2023).

  7. 7.

    By ‘knowledge’ we mean a form of ‘actionable information’—that is, information that can be used for making plans and getting things done in a particular environment.

  8. 8.

    The AERA system, for example, learned to do a TV-style interview after learning for only 20 h on a 6-core office desktop machine [35].

References

  1. What is the carbon footprint of a laptop? Circular Computing (2021). https://circularcomputing.com/news/carbon-footprint-laptop/

  2. Altman, S.: Moore’s law for everything (2021). https://moores.samaltman.com/

  3. Ananthaswamy, A.: In AI, is bigger always better? Nature 615, 202–205 (2023)

    Article  Google Scholar 

  4. Bender, E.M., McMillan-Major, A., Gebru, T., Shmitchell, S.: On the dangers of stochastic parrots: can language models be too big?. In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pp. 610–623 (2021)

    Google Scholar 

  5. Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., et al.: Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1–25 (2020)

    Google Scholar 

  6. Butler, S.: From retail to transport: how ai is changing every corner of the economy. The Guardian (2023). https://www.theguardian.com/technology/2023/feb/18/from-retail-to-transport-how-ai-is-changing-every-corner-of-the-economy

  7. Eberding, L.M., Thórisson, K.R., Prabu, A., Jaroria, S., Sheikhlar, A.: Comparison of machine learners on an aba experiment format of the cart-pole task. In: Proceedings of Machine Learning Research, vol. 159, pp. 49–63 (2021)

    Google Scholar 

  8. Hammer, P.: Reasoning-learning systems based on non-axiomatic reasoning system theory, vol. 192, pp. 88–106 (2022)

    Google Scholar 

  9. Hunt, E.: Tay, microsoft’s AI chatbot, gets a crash course in racism from twitter. The Guardian (2016). https://www.theguardian.com/technology/2016/mar/24/tay-microsofts-ai-chatbot-gets-a-crash-course-in-racism-from-twitter

  10. Kaplan, J., et al.: Scaling laws for neural language models (2020)

    Google Scholar 

  11. Klinger, J., Mateos-Garcia, J.C., Stathoulopoulos, K.: A narrowing of AI research? SSRN Electron. J. (2020)

    Google Scholar 

  12. Kommrusch, S., Minsky, H., Minsky, M., Shaoul, C.: Self-supervised learning for multi-goal grid world: Comparing leela and deep q network. In: Proceedings of Machine Learning Research, vol. 131, pp. 81–97 (2020)

    Google Scholar 

  13. Latapie, H., Gabriel, M., Kompella, R.: Hybrid AI for IoT actionable insights & real-time data-driven networks. In: Proceedings of Machine Learning Research, vol. 159, pp. 127–131 (2022)

    Google Scholar 

  14. Lenat, D.: Cyc: a large-scale investment in knowledge infrastructure. Commun. ACM 38, 33–38 (2023)

    Article  Google Scholar 

  15. Logunova, I.: A guide to one-shot learning (2022). https://serokell.io/blog/nn-and-one-shot-learning

  16. Luccioni, A.S., Viguier, S., Ligozat, A.L.: Estimating the carbon footprint of bloom, a 176b parameter language model (2022). https://arxiv.org/abs/2211.02001

  17. Marcus, G.: Deep learning is hitting a wall. Nautilus (2023). https://nautil.us/deep-learning-is-hitting-a-wall-238440/

  18. Marcus, G., Davis, E.: How not to test gpt-3 (2023). https://garymarcus.substack.com/p/how-not-to-test-gpt-3

  19. Marr, B.: 13 mind-blowing things artificial intelligence can already do today. Forbes (2019). https://www.forbes.com/sites/bernardmarr/2019/11/11/13-mind-blowing-things-artificial-intelligence-can-already-do-today/?sh=39e42add6502

  20. Maslej, N., Fattorini, L., Brynjolfsson, E., et al.: Artificial Intelligence Index Report 2023. Technical Report, Stanford University (2023). https://aiindex.stanford.edu/report/

  21. Mok, A.: AI is expensive. a search on Google’s chatbot Bard costs the company 10 times more than a regular one, which could amount to several billion dollars. Business Insider (2023). https://www.businessinsider.com/ai-expensive-google-chatbot-bard-may-cost-company-billions-dollars-2023-2

  22. Nielsen, K.H.: Vindmøllens historie: Sådan tæmmede danskerne vindens energi. ForskerZonen (2018). https://videnskab.dk/forskerzonen/teknologi-innovation/vindmoellenshistorie-saadan-taemmede-danskerne-vindens-energi

  23. Nivel, E., Thórisson, K.R., et al.: Autonomous acquisition of natural language. In: IADIS International Conference on Intelligent Systems & Agents, pp. 58–66 (2014)

    Google Scholar 

  24. Nivel, E., Thórisson, K.R., Steunebrink, B., Dindo, H., et al., G.P.: Autocatalytic endogenous reflective architecture. Tech report RUTR-SCS13002, Reykjavik University - School of Computer Science (2013)

    Google Scholar 

  25. Panch, T., Mattie, H., Celi, L.A.: The ‘inconvenient truth’ about AI in healthcare. NPJ Dig. Med. 2, 77 (2019)

    Article  Google Scholar 

  26. Pollock, J.L.: Defeasible reasoning and degrees of justification. Argu. Comput. 1(1), 7–22 (2010)

    Article  Google Scholar 

  27. Ritchie, H., Roser, M., Rosado, P.: Energy. Our World in Data (2022). https://ourworldindata.org/energy-mix

  28. Roose, K.: A conversation with Bing’s chatbot left me deeply unsettled. The New York Times (2023). https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html

  29. Sheikhlar, A., Thórisson, K.R., Eberding, L.M.: Autonomous cumulative transfer learning. In: International Conference on Artificial General Intelligence, pp. 306–316 (2020)

    Google Scholar 

  30. Sheikhlar, A., Thórisson, K.R., Thompson, J.: Explicit analogy for autonomous transversal learning, vol. 192, pp. 48–62 (2022)

    Google Scholar 

  31. Steunebrink, B., Swan, J., Nivel, E.: The Holon system: artificial general intelligence as ‘work on command’. In: Proceedings of Machine Learning Research, vol. 192, pp. 120–126 (2022)

    Google Scholar 

  32. Stojnic, G., Gandhi, K., Yasuda, S., Lake, B.M., Dillon, M.R.: Commonsense psychology in human infants and machines. Cognition 235, 105406 (2023)

    Article  Google Scholar 

  33. Strubell, E., Ganesh, A., McCallum, A.: Energy and policy considerations for deep learning in NLP. In: 57th Annual Meeting of the Association for Computational Linguistics (ACL) (2019)

    Google Scholar 

  34. Thórisson, K.R.: A new constructivist AI: from manual construction to self-constructive systems. In: Wang, P., Goertzel, B. (eds.) Theoretical Foundations of Artificial General Intelligence, vol. 4, pp. 145–171 (2012)

    Google Scholar 

  35. Thórisson, K.R.: Seed-programmed autonomous general learning. In: Proceedings of Machine Learning Research, vol. 131, pp. 32–70 (2020)

    Google Scholar 

  36. Thórisson, K.R., Bieger, J., Li, X., Wang, P.: Cumulative learning. In: Hammer, P., Agrawal, P., Goertzel, B., Iklé, M. (eds.) AGI 2019. LNCS (LNAI), vol. 11654, pp. 198–208. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-27005-6_20

    Chapter  Google Scholar 

  37. Thórisson, K.R., Minsky, H.: The future of AI research: ten defeasible ‘axioms of intelligence’. In: Proceedings of Machine Learning Research, vol. 192, pp. 5–21 (2022)

    Google Scholar 

  38. Thórisson, K.R.: The ‘Explanation Hypothesis’ in general self-supervised Learning. In: Proceedings of Machine Learning Research, IWSSL-21, pp. 5–27 (2021)

    Google Scholar 

  39. Wang, P.: Rigid Flexibility: The Logic of Intelligence, vol. 34. Springer, Heidelberg (2006). https://doi.org/10.1007/1-4020-5045-3

  40. Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models (2023)

    Google Scholar 

Download references

Acknowledgment

This work was supported in part by Cisco Systems, the Icelandic Institute for Intelligent Machines and Reykjavik University.

Author information

Authors and Affiliations

  1. Center for Analysis and Design of Intelligent Agents, Reykjavik University, Menntavegur 1, Reykjavík, Iceland

    Amanda Vallentin & Kristinn R. Thórisson

  2. IT University of Copenhagen, Rued Langgaards Vej 7, Copenhagen, Denmark

    Amanda Vallentin

  3. Icelandic Institute for Intelligent Machines, Reykjavík, Iceland

    Kristinn R. Thórisson

  4. Cisco Systems, Emerging Technologies and Incubation, San Jose, CA, USA

    Hugo Latapie

Authors
  1. Amanda Vallentin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristinn R. Thórisson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hugo Latapie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Amanda Vallentin , Kristinn R. Thórisson or Hugo Latapie .

Editor information

Editors and Affiliations

  1. Department of Psychology, Stockholm University, Stockholm, Sweden

    Patrick Hammer

  2. Örebro University, Örebro, Sweden

    Marjan Alirezaie

  3. University of Gothenburg, Gothenburg, Sweden

    Claes Strannegård

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Vallentin, A., Thórisson, K.R., Latapie, H. (2023). Addressing the Unsustainability of Deep Neural Networks with Next-Gen AI. In: Hammer, P., Alirezaie, M., Strannegård, C. (eds) Artificial General Intelligence. AGI 2023. Lecture Notes in Computer Science(), vol 13921. Springer, Cham. https://doi.org/10.1007/978-3-031-33469-6_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33469-6_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33468-9

  • Online ISBN: 978-3-031-33469-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation