Skip to main content
Springer Nature Link
Log in

The Case for an Ethical Black Box

  • Conference paper
  • First Online:
Towards Autonomous Robotic Systems (TAROS 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10454))

Included in the following conference series:

Abstract

This paper proposes that robots and autonomous systems should be equipped with the equivalent of a Flight Data Recorder to continuously record sensor and relevant internal status data. We call this an ethical black box. We argue that an ethical black box will be critical to the process of discovering why and how a robot caused an accident, and thus an essential part of establishing accountability and responsibility. We also argue that without the transparency afforded by an ethical black box, robots and autonomous systems are unlikely to win public trust.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.kurzweilai.net/googles-self-driving-car-gathers-nearly-1-gbsec.

  2. 2.

    https://standards.ieee.org/develop/project/7001.html.

References

  1. Allen, C., Smit, I., Wallach, W.: Artificial morality: top-down, bottom-up, and hybrid approaches. Ethics Inf. Technol. 7, 149–155 (2005)

    Article  Google Scholar 

  2. Anderson, M., Anderson, S.L.: GenEth: a general ethical dilemma analyzer. In: Proceedings of Twenty-Eighth AAAI Conference on Artificial Intelligence, pp. 253–261 (2014)

    Google Scholar 

  3. Arkin, R.C., Ulam, P., Wagner, A.R.: Moral decision making in autonomous systems: enforcement, moral emotions, dignity, trust, and deception. Proc. IEEE 100(3), 571–589 (2012)

    Article  Google Scholar 

  4. Boden, M., Bryson, J., Caldwell, D., Dautenhahn, K., Edwards, L., Kember, S., Newman, P., Parry, V., Pegman, G., Rodden, T., Sorrell, T., Wallis, M., Whitby, B., Winfield, A.F.: Principles of robotics. Connection Sci. 29(2), 124–129 (2017)

    Article  Google Scholar 

  5. Bonnefon, J.-F., Shariff, A., Rahwam, I.: The social dilemma of autonomous vehicles. Sci. 352(6293), 1573–1576 (2016)

    Google Scholar 

  6. British Standards Institute: BS8611: 2016 Robots and robotic devices: guide to the ethical design and application of robots and robotic systems, BSI London (2016). ISBN 9780580895302

    Google Scholar 

  7. Campbell, N.: The evolution of flight data analysis. In: Proceedings of Australian Society of Air Safety Investigators (2007). http://asasi.org/papers/2007/The_Evolution_of_Flight_Data_Analysis_Neil_Campbell.pdf

  8. Choudhuri, S., Givargis, T.: FlashBox: a system for logging non-deterministic events in deployed embedded systems. In: Proceedings of 2009 ACM Symposium on Applied Computing (SAC 2009), pp. 1676–1682 (2009)

    Google Scholar 

  9. Elbaum, S., Munson, J.C.: Software black box: an alternative mechanism for failure analysis. In: Proceedings of 11th International Symposium on Software Reliability Engineering, ISSRE 2000, pp. 365–376 (2000)

    Google Scholar 

  10. Hibbard, B.: Ethical artificial intelligence, arXiv:1411.1373v9 (2014)

  11. Caliskan-Islam, A., Bryson, J., Narayanan, A.: Semantics derived automatically from language corpora necessarily contain human biases, arXiv:1608.07187v2 (2016)

  12. Carroll, A., Heiser, G.: An analysis of power consumption in a smartphone. In: Proceedings of 2010 USENIX Annual Technical Conference, Boston, June 2010

    Google Scholar 

  13. Dennis, L.A., Fisher, M., Slavkovik, M., Webster, M.: Formal verification of ethical choices in autonomous systems. Robot. Auton. Syst. 77, 1–14 (2016)

    Article  Google Scholar 

  14. Dillmann, R.: Benchmarks for robotics research, EURON, April 2004. http://www.cas.kth.se/euron/euron-deliverables/ka1-10-benchmarking.pdf

  15. Fisher, M., List, C., Slavkovik, M., Winfield, A.F.: Engineering moral machines. In: Informatik-Spektrum. Springer, Berlin (2016)

    Google Scholar 

  16. Gibson, R.W.: IEEE 802 standards efforts. Comput. Netw. ISDN Syst. 19(2), 95–104 (1990)

    Article  Google Scholar 

  17. Grossi, D.R.: Aviation recorder overview, National Transportation Safety Board [NTSB]. J. Accid. Investig. 2(1), 31–42 (2006)

    Google Scholar 

  18. IEEE: Global initiative on Ethical Considerations in the Design of Artificial Intelligence and Autonomous Systems (2016). http://standards.ieee.org/develop/indconn/ec/autonomous_systems.html

  19. ISO 13482:2014 Robots and robotic devices Safety requirements for personal care robots (2014). http://www.iso.org/iso/catalogue_detail.htm?csnumber=53820

  20. Kavi, K.M.: Beyond the Black Box, IEEE Spectrum, Posted 30. http://spectrum.ieee.org/aerospace/aviation/beyond-the-black-box

  21. Menig, P., Coverdill, C.: Transportation recorders on commercial vehicles. In: Proceedings of 1999 International Symposium on Transportation Recorders (1999)

    Google Scholar 

  22. Moog, R.A.: MIDI: Musical Instrument Digital Interface. J. Audio Eng. Soc. 34(5), 394–404 (1986)

    Google Scholar 

  23. Perez, A., Garca, M.I., Nieto, M., Pedraza, J.L., Rodrguez, S., Zamorano, J.: Argos: an advanced in-vehicle data recorder on a massively sensorized vehicle for car driver behavior experimentation. IEEE Trans. Intell. Transp. Syst. 11(2), 463–473 (2010)

    Google Scholar 

  24. National Highway Traffic Safety Administration: Investigation Report PE 16–007 (2017). https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF

  25. Vlasic, B., Boudette, N.E.: Self-Driving Tesla Was Involved in Fatal Crash, U.S. Says, New York Times, 30 June 2016

    Google Scholar 

  26. Palmerini, E., Azzarri, F., Battaglia, A., Bertolini, A., Carnevale, A., Carpaneto, J., Cavallo, F., Di Carlo, A., Cempini, M., Controzzi, M., Koops, B.J., Lucivero, F., Mukerji, N., Nocco, L., Pirni, A., Shah, H., Salvini, P., Schellekens, M., Warwick, K.: D6.2 Guidelines on regulating robotics, Robolaw project (2014). http://www.robolaw.eu/RoboLaw_files/documents/robolaw_d6.2_guidelinesregulatingrobotics_20140922.pdf

  27. The Rome Declaration on Responsible Research and Innovation (2014). http://www.science-and-you.com/en/sis-rri-conference-recommendations-rome-declaration-responsible-research-and-innovation

  28. Thom, P.R., MacCarley, C.A.: A spy under the hood: controlling risk and automotive EDR. Risk Manag. Mag. 55(2), 22–26 (2008)

    Google Scholar 

  29. Sharabayko, M.P., Markov, N.G.: H.264/AVC video compression on smartphones. J. Phys. Conf. Seri. 803(1) (2017). Aricle No. 012141

    Google Scholar 

  30. Stilgoe, J.: Tesla crash report blames human error - this is a missed opportunity, The Guardian, 21 January 2017. https://www.theguardian.com/science/political-science/2017/jan/21/tesla-crash-report-blames-human-error-this-is-a-missed-opportunity

  31. Vladeck, D.C.: Machines without principals: liability rules and artificial intelligence. Wash. L. Rev. 89(117), 117–150 (2014)

    Google Scholar 

  32. Wilsdon, J., Willis, R.: See-through science: why public engagement needs to move upstream, DEMOS (2014)

    Google Scholar 

  33. Winfield, A.F.: Robotics: A Very Short Introduction, Oxford University Press, Oxford (2012)

    Google Scholar 

  34. Winfield, A.F.T., Blum, C., Liu, W.: Towards an ethical robot: internal models, consequences and ethical action selection. In: Mistry, M., Leonardis, A., Witkowski, M., Melhuish, C. (eds.) TAROS 2014. LNCS, vol. 8717, pp. 85–96. Springer, Cham (2014). doi:10.1007/978-3-319-10401-0_8

    Google Scholar 

  35. Winfield, A.F: Written evidence submitted to the UK Parliamentary Select Committee on Science and Technology Inquiry on Robotics and Artificial Intelligence, Discussion Paper, Science and Technology Committee (Commons) (2016)

    Google Scholar 

  36. Worrell, M: Analysis of Bruntingthorpe crash test data, impact. J. Inst. Traffic Accid. Investigators 21(1), 4–10 (2016)

    Google Scholar 

Download references

Acknowledgments

This work has, in part, been supported by EPSRC grant ref EP/L024861/1. We are also grateful to the anonymous reviewers for their insightful comments.

Author information

Authors and Affiliations

  1. Bristol Robotics Lab, University of the West of England, Bristol, England

    Alan F. T. Winfield

  2. Department of Computer Science, University of Oxford, Oxford, England

    Marina Jirotka

Authors
  1. Alan F. T. Winfield
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marina Jirotka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alan F. T. Winfield .

Editor information

Editors and Affiliations

  1. University of Surrey, Guildford, United Kingdom

    Yang Gao

  2. University of Surrey, Guildford, United Kingdom

    Saber Fallah

  3. University of Surrey, Guildford, United Kingdom

    Yaochu Jin

  4. University of Surrey, Guildford, United Kingdom

    Constantina Lekakou

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Winfield, A.F.T., Jirotka, M. (2017). The Case for an Ethical Black Box. In: Gao, Y., Fallah, S., Jin, Y., Lekakou, C. (eds) Towards Autonomous Robotic Systems. TAROS 2017. Lecture Notes in Computer Science(), vol 10454. Springer, Cham. https://doi.org/10.1007/978-3-319-64107-2_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64107-2_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64106-5

  • Online ISBN: 978-3-319-64107-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics