Skip to main content
SpringerLink
Log in

Safety Constraints and Ethical Principles in Collective Decision Making Systems

  • Conference paper
  • First Online:
KI 2015: Advances in Artificial Intelligence (KI 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9324))

Abstract

The future will see autonomous machines acting in the same environment as humans, in areas as diverse as driving, assistive technology, and health care. Think of self-driving cars, companion robots, and medical diagnosis support systems. We also believe that humans and machines will often need to work together and agree on common decisions. Thus hybrid collective decision making systems will be in great need. In this scenario, both machines and collective decision making systems should follow some form of moral values and ethical principles (appropriate to where they will act but always aligned to humans’), as well as safety constraints. In fact, humans would accept and trust more machines that behave as ethically as other humans in the same environment. Also, these principles would make it easier for machines to determine their actions and explain their behavior in terms understandable by humans. Moreover, often machines and humans will need to make decisions together, either through consensus or by reaching a compromise. This would be facilitated by shared moral values and ethical principles.

This extended abstract describes a research project funded by the Future of Life Institute, 2015-2018.

Joint work with J. Greene (Harvard Univ.), J. Tasioulas (King’s College London), K. B. Venable (Tulane Univ. and IMHC), B. Williams (MIT).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Arrow, K.J., Sen, A.K., Suzumara, K.: Handbook of Social Choice and Welfare. North-Holland, Elsevier (2002)

    Google Scholar 

  2. Bemporad, A., Morari, M.: Robust model-predictive control: a survey. In: Garulli, A., Tesi, A. (eds.) Robustness in Identification and Control. LNCS, vol. 245, pp. 207–226. Springer, London (1999)

    Chapter  Google Scholar 

  3. Bertsekas, D.P.: Dynamic Programming and Stochastic Control. Academic Press (1976)

    Google Scholar 

  4. Bistarelli, S., Pini, M.S., Rossi, F., Venable, K.B.: Bipolar preference problems: framework, properties and solving techniques. In: Azevedo, F., Barahona, P., Fages, F., Rossi, F. (eds.) CSCLP. LNCS (LNAI), vol. 4651, pp. 78–92. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  5. Bostrom, N.: Superintelligence: Paths, Dangers, Strategies. Oxford University Press (2014)

    Google Scholar 

  6. Boutilier, C., Brafman, R.I., Domshlak, C., Hoos, H.H., Poole, D.: CP-nets: A tool for representing and reasoning with conditional ceteris paribus preference statements. J. Artif. Intell. Res. (JAIR) 21, 135–191 (2004)

    MathSciNet  MATH  Google Scholar 

  7. Dechter, R.: Constraint Processing. Morgan Kaufmann (2003)

    Google Scholar 

  8. Fang, C., Yu, P., Williams, B.C.: Chance-constrained probabilistic simple temporal problems. In: Proc. AAAI 2014. AAAI Press (2014)

    Google Scholar 

  9. Grandi, U., Luo, H., Maudet, N., Rossi, F.: Aggregating CP-nets with unfeasible outcomes. In: O’Sullivan, B. (ed.) CP 2014. LNCS, vol. 8656, pp. 366–381. Springer, Heidelberg (2014)

    Google Scholar 

  10. Greene, J.D.: The cognitive neuroscience of moral judgment and decision-making. MIT Press (2014)

    Google Scholar 

  11. Meseguer, P., Rossi, F., Schiex, T.: Soft constraints. In: Handbook of constraint programming, chapter 9, pp. 281–328. Elsevier (2006)

    Google Scholar 

  12. Moor, J.H.: What is computer ethics? Metaphilosophy 16(4), 266–275 (1985)

    Article  Google Scholar 

  13. Ono, M., Blackmore, L., Williams, B.: Chance constrained finite horizon optimal control with nonconvex constraints. In: Proc. American Control Conference (2010)

    Google Scholar 

  14. Prekopa, A.: Stochastic Programming. Kluwer (1995)

    Google Scholar 

  15. Puterman, M.L.: Markov Decision Processes. Wiley (1994)

    Google Scholar 

  16. Rossi, F., Van Beek, Walsh, T.: Handbook of Constraint Programming. Elsevier (2006)

    Google Scholar 

  17. Rossi, F., Venable, K.B., Walsh, T.: A Short introduction to preferences: between artificial intelligence and social choice. In: Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers (2011)

    Google Scholar 

  18. Santana, P.H.R.Q.A., Williams, B.C.: Chance-constrained consistency for probabilistic temporal plan networks. In: Proc. AAAI 2014. AAAI Press (2014)

    Google Scholar 

  19. Schwarm, A., Nikolaou, M.: Chance-constrained model-predictive control. AlChE Journal 45, 1743–1752 (1999)

    Article  Google Scholar 

  20. Shenhav, A., Greene, J.D.: Integrative moral judgment: dissociating the roles of the amygdala and ventromedial prefrontal cortex. The Journal of Neuroscience 34(13), 4741–4749 (2014)

    Article  Google Scholar 

  21. Thomson, J.J.: The trolley problem. Yale Law Journal 94, 1395 (1985)

    Article  Google Scholar 

  22. Vidal, T., Fargier, H.: Handling contingency in temporal constraint networks: from consistency to controllabilities. Journal of Experimental and Theoretical Artificial Intelligence 11, 23–45 (1999)

    Article  MATH  Google Scholar 

  23. Wallach, W., Allen, C.: Moral machines: Teaching robots right from wrong. Oxford University Press (2008)

    Google Scholar 

  24. Yu, P., Fang, C., Williams, B.C.: Resolving over-constrained probabilistic temporal problems through chance constraint relaxation. In: Proc. AAAI 2015. AAAI Press (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Padova, Padua, Italy

    Francesca Rossi

Authors
  1. Francesca Rossi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francesca Rossi .

Editor information

Editors and Affiliations

  1. Technische Universität Dresden, Dresden, Germany

    Steffen Hölldobler

  2. Technische Universität Dresden, Dresden, Germany

  3. Free University Bozen-Bolzano, Bozen-Bolzano, Italy

    Rafael Peñaloza

  4. Technische Universität Dresden, Dresden, Germany

    Sebastian Rudolph

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Rossi, F. (2015). Safety Constraints and Ethical Principles in Collective Decision Making Systems. In: Hölldobler, S., , Peñaloza, R., Rudolph, S. (eds) KI 2015: Advances in Artificial Intelligence. KI 2015. Lecture Notes in Computer Science(), vol 9324. Springer, Cham. https://doi.org/10.1007/978-3-319-24489-1_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24489-1_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24488-4

  • Online ISBN: 978-3-319-24489-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation