research-article

Two Heads Are Better Than One: A Dimension Space for Unifying Human and Artificial Intelligence in Shared Control

Authors:

Gabriele Cimolino,

T.C. Nicholas GrahamAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 8, Pages 1 - 21

https://doi.org/10.1145/3491102.3517610

Published: 28 April 2022 Publication History

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Two Heads Are Better Than One: A Dimension Space for Unifying Human and Artificial Intelligence in Shared Control

Pages 1 - 21

Abstract
Supplemental Material
References

Abstract

Shared control is an emerging interaction paradigm in which a human and an AI partner collaboratively control a system. Shared control unifies human and artificial intelligence, making the human’s interactions with computers more accessible, safe, precise, effective, creative, and playful. This form of interaction has independently emerged in contexts as varied as mobility assistance, driving, surgery, and digital games. These domains each have their own problems, terminology, and design philosophies. Without a common language for describing interactions in shared control, it is difficult for designers working in one domain to share their knowledge with designers working in another. To address this problem, we present a dimension space for shared control, based on a survey of 55 shared control systems from six different problem domains. This design space analysis tool enables designers to classify existing systems, make comparisons between them, identify higher-level design patterns, and imagine solutions to novel problems.

Supplemental Material

MP4 File

Talk Video

Transcript for: Talk Video

References

[1]

David A. Abbink, Tom Carlson, Mark Mulder, Joost C. F. de Winter, Farzad Aminravan, Tricia L. Gibo, and Erwin R. Boer. 2018. A Topology of Shared Control Systems—Finding Common Ground in Diversity. IEEE Transactions on Human-Machine Systems 48, 5 (Oct. 2018), 509–525. https://doi.org/10.1109/THMS.2018.2791570 Conference Name: IEEE Transactions on Human-Machine Systems.

[2]

David A. Abbink, Mark Mulder, and Erwin R. Boer. 2012. Haptic shared control: smoothly shifting control authority?Cognition, Technology & Work 14, 1 (March 2012), 19–28. https://doi.org/10.1007/s10111-011-0192-5

Digital Library

[3]

Christopher Alexander. 1979. The timeless way of building. Oxford University Press, New York.

[4]

Christopher Alexander, Sara Ishikawa, and Murray Silverstein. 1977. A pattern language: towns, buildings, construction. Oxford University Press, New York.

[5]

J.E. Allen, C.I. Guinn, and E. Horvtz. 1999. Mixed-initiative interaction. IEEE Intelligent Systems and their Applications 14, 5 (Sept. 1999), 14–23. https://doi.org/10.1109/5254.796083 Conference Name: IEEE Intelligent Systems and their Applications.

Digital Library

[6]

Edoardo Aromataris and Zachary Munn (Eds.). 2020. JBI Manual for Evidence Synthesis. JBI. https://doi.org/10.46658/JBIMES-20-01

[7]

Erasmo Artur and Rosane Minghim. 2019. A novel visual approach for enhanced attribute analysis and selection. Computers & Graphics 84 (Nov. 2019), 160–172. https://doi.org/10.1016/j.cag.2019.08.015

Digital Library

[8]

Lisanne Bainbridge. 1983. Ironies of automation. Automatica 19, 6 (1983), 775–779. https://doi.org/10.1016/0005-1098(83)90046-8

Digital Library

[9]

Marcel Caspar Attila Baltzer, Daniel López, and Frank Flemisch. 2019. Towards an interaction pattern language for human machine cooperation and cooperative movement. Cognition, Technology & Work 21, 4 (Nov. 2019), 593–606. https://doi.org/10.1007/s10111-019-00561-8

Digital Library

[10]

Scott Bateman, Regan L. Mandryk, Tadeusz Stach, and Carl Gutwin. 2011. Target assistance for subtly balancing competitive play. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11. ACM Press, Vancouver, BC, Canada, 2355. https://doi.org/10.1145/1978942.1979287

Digital Library

[11]

D. Birnbaum, R. Fiebrink, J. Malloch, and M. M. Wanderley. 2005. Towards a dimension space for musical devices. In Proceedings of the 2005 conference on New interfaces for musical expression(NIME ’05). National University of Singapore, SGP, 192–195.

[12]

Robin N. Brewer and Vaishnav Kameswaran. 2018. Understanding the Power of Control in Autonomous Vehicles for People with Vision Impairment. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility(ASSETS ’18). Association for Computing Machinery, New York, NY, USA, 185–197. https://doi.org/10.1145/3234695.3236347

Digital Library

[13]

Jared E. Cechanowicz, Carl Gutwin, Scott Bateman, Regan Mandryk, and Ian Stavness. 2014. Improving player balancing in racing games. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play(CHI PLAY ’14). Association for Computing Machinery, New York, NY, USA, 47–56. https://doi.org/10.1145/2658537.2658701

Digital Library

[14]

Gabriele Cimolino, Sussan Askari, and T.C. Nicholas Graham. 2021. The Role of Partial Automation in Increasing the Accessibility of Digital Games. In Proceedings of the Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’21). ACM, New York, NY, USA.

Digital Library

[15]

Sylvain Daronnat. 2020. Human-Agent Trust Relationships in a Real-Time Collaborative Game. In Extended Abstracts of the 2020 Annual Symposium on Computer-Human Interaction in Play(CHI PLAY ’20). Association for Computing Machinery, New York, NY, USA, 18–20. https://doi.org/10.1145/3383668.3419953

Digital Library

[16]

Sylvain Daronnat, Leif Azzopardi, and Martin Halvey. 2020. Impact of agents’ errors on performance, reliance and trust in human-agent collaboration. 1–5. https://www.hfes2020.com/

[17]

Sylvain Daronnat, Leif Azzopardi, Martin Halvey, and Mateusz Dubiel. 2020. Impact of Agent Reliability and Predictability on Trust in Real Time Human-Agent Collaboration. In Proceedings of the 8th International Conference on Human-Agent Interaction(HAI ’20). Association for Computing Machinery, New York, NY, USA, 131–139. https://doi.org/10.1145/3406499.3415063

Digital Library

[18]

Xiaoyan Deng, Zhu Liang Yu, Canguang Lin, Zhenghui Gu, and Yuanqing Li. 2020. Self-adaptive shared control with brain state evaluation network for human-wheelchair cooperation. Journal of Neural Engineering 17, 4 (July 2020), 045005. https://doi.org/10.1088/1741-2552/ab937e Publisher: IOP Publishing.

[19]

DJI. [n. d.]. Mavic Air 2 - Up Your Game. Retrieved 2022-01-10 from https://www.dji.com/ca/mavic-air-2?site=brandsite&from=nav

[20]

Anca D. Dragan, Kenton C.T. Lee, and Siddhartha S. Srinivasa. 2013. Legibility and predictability of robot motion. In 2013 8th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 301–308. https://doi.org/10.1109/HRI.2013.6483603 ISSN: 2167-2148.

[21]

Anca D. Dragan and Siddhartha S. Srinivasa. 2012. Assistive teleoperation for manipulation tasks. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction(HRI ’12). Association for Computing Machinery, Boston, Massachusetts, USA, 123–124. https://doi.org/10.1145/2157689.2157716

Digital Library

[22]

Anca D Dragan and Siddhartha S Srinivasa. 2013. A policy-blending formalism for shared control. The International Journal of Robotics Research 32, 7 (June 2013), 790–805. https://doi.org/10.1177/0278364913490324

Digital Library

[23]

Nintendo EAD. 2017. Mario Kart 8 Deluxe. Game [Nintendo Switch]. Nintendo, Kyoto, Japan.

[24]

Jakob Engel, Jürgen Sturm, and Daniel Cremers. 2014. Scale-aware navigation of a low-cost quadrocopter with a monocular camera. Robotics and Autonomous Systems 62, 11 (Nov. 2014), 1646–1656. https://doi.org/10.1016/j.robot.2014.03.012

[25]

Chinemelu Ezeh, Pete Trautman, Louise Devigne, Valentin Bureau, Marie Babel, and Tom Carlson. 2017. Probabilistic vs linear blending approaches to shared control for wheelchair driving. In 2017 International Conference on Rehabilitation Robotics (ICORR). 835–840. https://doi.org/10.1109/ICORR.2017.8009352 ISSN: 1945-7901.

Digital Library

[26]

Chinemelu Ezeh, Pete Trautman, Catherine Holloway, and Tom Carlson. 2017. Comparing shared control approaches for alternative interfaces: A wheelchair simulator experiment. In 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 93–98. https://doi.org/10.1109/SMC.2017.8122584

Digital Library

[27]

F. Flemisch, D. Abbink, M. Itoh, M-P. Pacaux-Lemoine, and G. Weßel. 2016. Shared control is the sharp end of cooperation: Towards a common framework of joint action, shared control and human machine cooperation. IFAC-PapersOnLine 49, 19 (Jan. 2016), 72–77. https://doi.org/10.1016/j.ifacol.2016.10.464

[28]

F. Flemisch, D. A. Abbink, M. Itoh, M.-P. Pacaux-Lemoine, and G. Weßel. 2019. Joining the blunt and the pointy end of the spear: towards a common framework of joint action, human–machine cooperation, cooperative guidance and control, shared, traded and supervisory control. Cognition, Technology & Work 21, 4 (Nov. 2019), 555–568. https://doi.org/10.1007/s10111-019-00576-1

Digital Library

[29]

Frank Flemisch, Johann Kelsch, Christan Löper, Anna Schieben, Julian Schindler, and Matthias Heesen. 2008. Cooperative Control and Active Interfaces for Vehicle Assitsance and Automation. München. https://elib.dlr.de/57618/

[30]

Frank Ole Flemisch, Klaus Bengler, Heiner Bubb, Hermann Winner, and Ralph Bruder. 2014. Towards cooperative guidance and control of highly automated vehicles: H-Mode and Conduct-by-Wire. Ergonomics 57, 3 (March 2014), 343–360. https://doi.org/10.1080/00140139.2013.869355 Publisher: Taylor & Francis _eprint: https://doi.org/10.1080/00140139.2013.869355.

[31]

Frank O. Adams Flemisch. 2003. The H-Metaphor as a Guideline for Vehicle Automation and Interaction. Technical Report. https://ntrs.nasa.gov/search.jsp?R=20040031835

[32]

Antonio Franchi, Cristian Secchi, Markus Ryll, Heinrich H. Bulthoff, and Paolo Robuffo Giordano. 2012. Shared Control : Balancing Autonomy and Human Assistance with a Group of Quadrotor UAVs. IEEE Robotics Automation Magazine 19, 3 (Sept. 2012), 57–68. https://doi.org/10.1109/MRA.2012.2205625

[33]

Anna-Katharina Frison, Philipp Wintersberger, Andreas Riener, and Clemens Schartmüller. 2017. Driving Hotzenplotz: A Hybrid Interface for Vehicle Control Aiming to Maximize Pleasure in Highway Driving. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications(AutomotiveUI ’17). Association for Computing Machinery, New York, NY, USA, 236–244. https://doi.org/10.1145/3122986.3123016

Digital Library

[34]

Milad Geravand, Christian Werner, Klaus Hauer, and Angelika Peer. 2016. An Integrated Decision Making Approach for Adaptive Shared Control of Mobility Assistance Robots. International Journal of Social Robotics 8, 5 (Nov. 2016), 631–648. https://doi.org/10.1007/s12369-016-0353-z

[35]

T. C. Nicholas Graham, Leon A. Watts, Gaëlle Calvary, Joëlle Coutaz, Emmanuel Dubois, and Laurence Nigay. 2000. A dimension space for the design of interactive systems within their physical environments. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques(DIS ’00). Association for Computing Machinery, New York, NY, USA, 406–416. https://doi.org/10.1145/347642.347799

Digital Library

[36]

Hadrien Gurnel, Maud Marchal, Laurent Launay, Luc Beuzit, and Alexandre Krupa. 2019. Design of haptic guides for pre-positioning assistance of a comanipulated needle. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). 478–485. https://doi.org/10.1109/SMC.2019.8914395 ISSN: 2577-1655.

Digital Library

[37]

Marco Antonio Gutierrez, Luis Fernando D’Haro, and Rafael Banchs. 2016. A Multimodal Control Architecture for Autonomous Unmanned Aerial Vehicles. In Proceedings of the Fourth International Conference on Human Agent Interaction(HAI ’16). Association for Computing Machinery, New York, NY, USA, 107–110. https://doi.org/10.1145/2974804.2980522

Digital Library

[38]

Ian Hattwick and Marcelo Wanderley. 2012. A Dimension Space for Evaluating Collaborative Musical Performance Systems. In Proceedings of the International Conference on New Interfaces for Musical Expression. University of Michigan, Ann Arbor, Michigan. https://doi.org/10.5281/zenodo.1178281

[39]

Eric Horvitz. 1999. Mixed-Initiative Interaction. IEEE Intelligent Systems (Sept. 1999), 14–24. https://www.microsoft.com/en-us/research/publication/mixed-initiative-interaction/ Edition: IEEE Intelligent Systems.

[40]

Susan Hwang, Adrian L. Jessup Schneider, Daniel Clarke, Alexander Macintosh, Lauren Switzer, Darcy Fehlings, and T.C. Nicholas Graham. 2017. How Game Balancing Affects Play: Player Adaptation in an Exergame for Children with Cerebral Palsy. In Proceedings of the 2017 Conference on Designing Interactive Systems - DIS ’17. ACM Press, Edinburgh, United Kingdom, 699–710. https://doi.org/10.1145/3064663.3064664

Digital Library

[41]

Toshiyuki Inagaki. 2003. Adaptive automation: Sharing and trading of control. In Handbook of cognitive task design. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US, 147–169. https://doi.org/10.1201/9781410607775.ch8

[42]

Vahid Izadi, Arjun Yeravdekar, and Amirhossein Ghasemi. 2019. Determination of Roles and Interaction Modes in a Haptic Shared Control Framework. American Society of Mechanical Engineers Digital Collection. https://doi.org/10.1115/DSCC2019-9042

[43]

Matjaž Jakopec, Simon J. Harris, Ferdinando Rodriguez y Baena, Paula Gomes, Justin Cobb, and Brian L. Davies. 2002. Preliminary Results of an Early Clinical Experience with the Acrobot™ System for Total Knee Replacement Surgery. In Medical Image Computing and Computer-Assisted Intervention — MICCAI 2002(Lecture Notes in Computer Science), Takeyoshi Dohi and Ron Kikinis (Eds.). Springer, Berlin, Heidelberg, 256–263. https://doi.org/10.1007/3-540-45786-0_32

[44]

Jingjing Jiang and Alessandro Astolfi. 2015. Shared-control for a UAV operating in the 3D space. In 2015 European Control Conference (ECC). 1633–1638. https://doi.org/10.1109/ECC.2015.7330771

[45]

Shu D. Jiang and Jonathan Odom. 2018. Toward Initiative Decision-Making for Distributed Human-Robot Teams. In Proceedings of the 6th International Conference on Human-Agent Interaction(HAI ’18). Association for Computing Machinery, New York, NY, USA, 286–292. https://doi.org/10.1145/3284432.3284467

Digital Library

[46]

Mishel Johns, Brian Mok, David Sirkin, Nikhil Gowda, Catherine Smith, Walter Talamonti, and Wendy Ju. 2016. Exploring shared control in automated driving. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI). 91–98. https://doi.org/10.1109/HRI.2016.7451738 ISSN: 2167-2148.

[47]

Gavin Kane, Georg Eggers, Robert Boesecke, Jörg Raczkowsky, Heinz Wörn, Rüdiger Marmulla, and Joachim Mühling. 2009. System Design of a Hand-Held Mobile Robot for Craniotomy. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009(Lecture Notes in Computer Science), Guang-Zhong Yang, David Hawkes, Daniel Rueckert, Alison Noble, and Chris Taylor (Eds.). Springer, Berlin, Heidelberg, 402–409. https://doi.org/10.1007/978-3-642-04268-3_50

Digital Library

[48]

Ankur Kapoor, Rajesh Kumar, and Russell H. Taylor. 2003. Simple Biomanipulation Tasks with “Steady Hand” Cooperative Manipulator. In Medical Image Computing and Computer-Assisted Intervention - MICCAI 2003(Lecture Notes in Computer Science), Randy E. Ellis and Terry M. Peters (Eds.). Springer, Berlin, Heidelberg, 141–148. https://doi.org/10.1007/978-3-540-39899-8_18

[49]

Shunichi Kasahara, Jun Nishida, and Pedro Lopes. 2019. Preemptive Action: Accelerating Human Reaction using Electrical Muscle Stimulation Without Compromising Agency. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–15. https://doi.org/10.1145/3290605.3300873

Digital Library

[50]

Soheil Kianzad, Yuxiang Huang, Robert Xiao, and Karon E. MacLean. 2020. Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376134

Digital Library

[51]

Hyunjung Kim, Seoktae Kim, Boram Lee, Jinhee Pak, Minjung Sohn, Geehyuk Lee, and Woohun Lee. 2008. Digital rubbing: playful and intuitive interaction technique for transferring a graphic image onto paper with pen-based computing. In CHI ’08 Extended Abstracts on Human Factors in Computing Systems(CHI EA ’08). Association for Computing Machinery, New York, NY, USA, 2337–2342. https://doi.org/10.1145/1358628.1358680

Digital Library

[52]

Holger Klapperich and Marc Hassenzahl. 2016. Hotzenplotz: Reconciling Automation with Experience. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction(NordiCHI ’16). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/2971485.2971532

Digital Library

[53]

Moritz Körber, Eva Baseler, and Klaus Bengler. 2018. Introduction matters: Manipulating trust in automation and reliance in automated driving. Applied Ergonomics 66 (Jan. 2018), 18–31. https://doi.org/10.1016/j.apergo.2017.07.006

[54]

Jung Min Lee and Da Young Ju. 2018. Classification of Human‐Vehicle Interaction: User Perspectives on Design. Social Behavior and Personality: an international journal 46, 7 (July 2018), 1057–1070. https://doi.org/10.2224/sbp.6242

[55]

M. P. Lemoine, S. Debernard, I. Crevits, and P. Millot. 1996. Cooperation between humans and machines: First results of an experiment with a multi-level cooperative organisation in air traffic control. Computer Supported Cooperative Work (CSCW) 5, 2 (June 1996), 299–321. https://doi.org/10.1007/BF00133661

Digital Library

[56]

Qinan Li, Weidong Chen, and Jingchuan Wang. 2011. Dynamic shared control for human-wheelchair cooperation. In 2011 IEEE International Conference on Robotics and Automation. 4278–4283. https://doi.org/10.1109/ICRA.2011.5980055 ISSN: 1050-4729.

[57]

Ana C. Lopes, Urbano Nunes, Luis Vaz, and Luís Vaz. 2010. Assisted navigation based on shared-control, using discrete and sparse human-machine interfaces. In 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology. 471–474. https://doi.org/10.1109/IEMBS.2010.5626221 ISSN: 1558-4615.

[58]

Ana C. Lopes, Gabriel Pires, Luís Vaz, and Urbano Nunes. 2011. Wheelchair navigation assisted by Human-Machine shared-control and a P300-based Brain Computer Interface. In 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2438–2444. https://doi.org/10.1109/IROS.2011.6094748 ISSN: 2153-0866.

[59]

Pedro Lopes, Alexandra Ion, Willi Mueller, Daniel Hoffmann, Patrik Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(CHI ’15). Association for Computing Machinery, New York, NY, USA, 939–948. https://doi.org/10.1145/2702123.2702461

Digital Library

[60]

Pedro Lopes, Doăa Yüksel, François Guimbretière, and Patrick Baudisch. 2016. Muscle-plotter: An Interactive System based on Electrical Muscle Stimulation that Produces Spatial Output. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology(UIST ’16). Association for Computing Machinery, New York, NY, USA, 207–217. https://doi.org/10.1145/2984511.2984530

Digital Library

[61]

Robert A. MacLachlan, Brian C. Becker, Jaime Cuevas Tabares, Gregg W. Podnar, Louis A. Lobes, and Cameron N. Riviere. 2012. Micron: An Actively Stabilized Handheld Tool for Microsurgery. IEEE Transactions on Robotics 28, 1 (Feb. 2012), 195–212. https://doi.org/10.1109/TRO.2011.2169634 Conference Name: IEEE Transactions on Robotics.

Digital Library

[62]

Allan Maclean and Diane McKerlie. 1995. Design space analysis and use representations. In Scenario-based design: envisioning work and technology in system development. John Wiley & Sons, Inc., USA, 183–207.

[63]

Lucas Medeiros and Flavio Coutinho. 2015. Developing an Accessible One-Switch Game for Motor Impaired Players. In Proceedings of SBGames 2015. Teresina, PI, Brazil, 236–239.

[64]

John A. Michon. 1985. A Critical View of Driver Behavior Models: What Do We Know, What Should We Do?In Human Behavior and Traffic Safety, Leonard Evans and Richard C. Schwing (Eds.). Springer US, Boston, MA, 485–524. https://doi.org/10.1007/978-1-4613-2173-6_19

[65]

P. Millot and A. Kamoun. 1988. An Implicit Method for Dynamic Task Allocation Between Man and Computer in Supervision Posts of Automated Processes. IFAC Proceedings Volumes 21, 5 (June 1988), 77–82. https://doi.org/10.1016/S1474-6670(17)53885-9

[66]

Patrick Millot and Marie-Pierre Pacaux-Lemoine. 2013. A Common Work Space for a mutual enrichment of Human-machine Cooperation and Team-Situation Awareness. IFAC Proceedings Volumes 46, 15 (Jan. 2013), 387–394. https://doi.org/10.3182/20130811-5-US-2037.00061

[67]

Selma Musić and Sandra Hirche. 2016. Classification of human-robot team interaction paradigms**The research leading to these results has received funding from the European Union Seventh Framework Programme FP7/2007- 2013 under grant agreement no. 601165 of the project: WEARHAP -WEARable HAPtics for humans and robots.IFAC-PapersOnLine 49, 32 (Jan. 2016), 42–47. https://doi.org/10.1016/j.ifacol.2016.12.187

[68]

Selma Musić and Sandra Hirche. 2017. Control sharing in human-robot team interaction. Annual Reviews in Control 44 (Jan. 2017), 342–354. https://doi.org/10.1016/j.arcontrol.2017.09.017

[69]

Jun Nishida, Shunichi Kasahara, and Kenji Suzuki. 2017. Wired muscle: generating faster kinesthetic reaction by inter-personally connecting muscles. In ACM SIGGRAPH 2017 Emerging Technologies(SIGGRAPH ’17). Association for Computing Machinery, New York, NY, USA, 1–2. https://doi.org/10.1145/3084822.3084844

Digital Library

[70]

Yoojin Oh, Shao-Wen Wu, Marc Toussaint, and Jim Mainprice. 2020. Natural Gradient Shared Control. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 1223–1229. https://doi.org/10.1109/RO-MAN47096.2020.9223465 ISSN: 1944-9437.

[71]

On-Road Automated Driving (ORAD) committee. 2021. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Standard J3016_202104. SAE International. 41 pages. https://doi.org/10.4271/J3016_202104

[72]

M. P. Pacaux-Lemoine and S. Debernard. 2000. A Common Work Space to Support the Air Traffic Control. IFAC Proceedings Volumes 33, 12 (June 2000), 75–78. https://doi.org/10.1016/S1474-6670(17)37280-4

[73]

Marie-Pierre Pacaux-Lemoine and Frank Flemisch. 2019. Layers of shared and cooperative control, assistance, and automation. Cognition, Technology & Work 21, 4 (Nov. 2019), 579–591. https://doi.org/10.1007/s10111-018-0537-4

Digital Library

[74]

Marie-Pierre Pacaux-Lemoine and Itoh Makoto. 2015. Towards Vertical and Horizontal Extension of Shared Control Concept. In 2015 IEEE International Conference on Systems, Man, and Cybernetics. 3086–3091. https://doi.org/10.1109/SMC.2015.536

Digital Library

[75]

R. Parasuraman, T.B. Sheridan, and C.D. Wickens. 2000. A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 30, 3 (May 2000), 286–297. https://doi.org/10.1109/3468.844354 Conference Name: IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

Digital Library

[76]

Tae Mun Park, Seung Yeon Won, Sang Ryong Lee, and Gabor Sziebig. 2016. Force feedback based gripper control on a robotic arm. In 2016 IEEE 20th Jubilee International Conference on Intelligent Engineering Systems (INES). 107–112. https://doi.org/10.1109/INES.2016.7555102

[77]

Christopher. J. Payne, Win Tun Latt, and Guang-Zhong Yang. 2012. A new hand-held force-amplifying device for micromanipulation. In 2012 IEEE International Conference on Robotics and Automation. 1583–1588. https://doi.org/10.1109/ICRA.2012.6225306 ISSN: 1050-4729.

[78]

Christopher J. Payne, Khushi Vyas, Daniel Bautista-Salinas, Dandan Zhang, Hani J. Marcus, and Guang-Zhong Yang. 2021. Shared-Control Robots. In Neurosurgical Robotics, Hani J. Marcus and Christopher J. Payne (Eds.). Springer US, New York, NY, 63–79. https://doi.org/10.1007/978-1-0716-0993-4_4

[79]

Bastian Pfleging, Maurice Rang, and Nora Broy. 2016. Investigating user needs for non-driving-related activities during automated driving. In Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia(MUM ’16). Association for Computing Machinery, New York, NY, USA, 91–99. https://doi.org/10.1145/3012709.3012735

Digital Library

[80]

William Pilgrim. 2008. Alienated. Game [Windows]. OneSwitch.org.uk, Essex, UK.

[81]

Sanzhar Rakhimkul, Anton Kim, Askarbek Pazylbekov, and Almas Shintemirov. 2019. Autonomous Object Detection and Grasping Using Deep Learning for Design of an Intelligent Assistive Robot Manipulation System. In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). 3962–3968. https://doi.org/10.1109/SMC.2019.8914465 ISSN: 2577-1655.

Digital Library

[82]

J. Rasmussen. 1983. Skills, rules, and knowledge; signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man, and Cybernetics SMC-13, 3 (May 1983), 257–266. https://doi.org/10.1109/TSMC.1983.6313160

[83]

Alec Rivers, Ilan E. Moyer, and Frédo Durand. 2012. Position-correcting tools for 2D digital fabrication. ACM Transactions on Graphics 31, 4 (July 2012), 88:1–88:7. https://doi.org/10.1145/2185520.2185584

Digital Library

[84]

Takuma Seno, Kohei Okuoka, Masahiko Osawa, and Michita Imai. 2018. Adaptive Semi-autonomous Agents via Episodic Control. In Proceedings of the 6th International Conference on Human-Agent Interaction(HAI ’18). Association for Computing Machinery, New York, NY, USA, 377–379. https://doi.org/10.1145/3284432.3287192

Digital Library

[85]

Chouki Sentouh, Anh-Tu Nguyen, Jrme Floris, and Jean-Christophe Popieul. 2018. Multiple Controller Switching Concept for Human-Machine Shared Control of Lane Keeping Assist Systems. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). 2730–2735. https://doi.org/10.1109/SMC.2018.00466 ISSN: 2577-1655.

Digital Library

[86]

Thomas B. Sheridan. 1992. Telerobotics, automation, and human supervisory control. MIT Press, Cambridge, Mass.

Digital Library

[87]

Thomas B. Sheridan and William L. Verplank. 1978. Human and Computer Control of Undersea Teleoperators. Technical Report. MASSACHUSETTS INST OF TECH CAMBRIDGE MAN-MACHINE SYSTEMS LAB. https://apps.dtic.mil/sti/citations/ADA057655 Section: Technical Reports.

[88]

Roy Shilkrot, Pattie Maes, Joseph A. Paradiso, and Amit Zoran. 2015. Augmented Airbrush for Computer Aided Painting (CAP). ACM Transactions on Graphics 34, 2 (March 2015), 19:1–19:11. https://doi.org/10.1145/2699649

Digital Library

[89]

Gillian Smith, Jim Whitehead, and Michael Mateas. 2010. Tanagra: a mixed-initiative level design tool. In Proceedings of the Fifth International Conference on the Foundations of Digital Games(FDG ’10). Association for Computing Machinery, New York, NY, USA, 209–216. https://doi.org/10.1145/1822348.1822376

Digital Library

[90]

Harold Soh and Yiannis Demiris. 2015. Learning assistance by demonstration: smart mobility with shared control and paired haptic controllers. Journal of Human-Robot Interaction 4, 3 (Dec. 2015), 76–100. https://doi.org/10.5898/JHRI.4.3.Soh

Digital Library

[91]

Micah Steele and R. Brent Gillespie. 2001. Shared Control between Human and Machine: Using a Haptic Steering Wheel to Aid in Land Vehicle Guidance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 45, 23 (Oct. 2001), 1671–1675. https://doi.org/10.1177/154193120104502323 Publisher: SAGE Publications Inc.

[92]

Stuart Morris, Gail Fawcett, Laurent Brisebois, and Jeffrey Hughes. 2018. A demographic, employment and income profile of Canadians with disabilities aged 15 years and over, 2017. Technical Report. Statistics Canada. https://www150.statcan.gc.ca/n1/pub/89-654-x/89-654-x2018002-eng.htm

[93]

Joshua P. Switkes, Eric J. Rossetter, Ian A. Coe, and J. Christian Gerdes. 2006. Handwheel Force Feedback for Lanekeeping Assistance: Combined Dynamics and Stability. Journal of Dynamic Systems, Measurement, and Control 128, 3 (Sept. 2006), 532–542. https://doi.org/10.1115/1.2229256

[94]

K.A. Tahboub. 2001. Natural and manmade shared-control systems: an overview. In Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164), Vol. 3. 2655–2660 vol.3. https://doi.org/10.1109/ROBOT.2001.933023 ISSN: 1050-4729.

[95]

Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). Association for Computing Machinery, New York, NY, USA, 543–552. https://doi.org/10.1145/1978942.1979018

Digital Library

[96]

Russell Taylor, Patrick Jensen, Louis Whitcomb, Aaron Barnes, Rajesh Kumar, Dan Stoianovici, Puneet Gupta, ZhengXian Wang, Eugene deJuan, and Louis Kavoussi. 1999. A Steady-Hand Robotic System for Microsurgical Augmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI’99(Lecture Notes in Computer Science), Chris Taylor and Alain Colchester (Eds.). Springer, Berlin, Heidelberg, 1031–1041. https://doi.org/10.1007/10704282_112

[97]

Shan-Yuan Teng, Rong-Hao Liang, Jo-Hsi Tang, Bing-Yu Chen, and Yi-Chi Liao. [n. d.]. Playing Air Guitar by Electrical Muscle Stimulation. AUI 2016: 1 st Asian Workshop on User Interface ([n. d.]), 2.

[98]

Tesla. 2018. Introducing Navigate on Autopilot. Retrieved 2022-01-10 from https://www.tesla.com/blog/introducing-navigate-autopilot

[99]

TIEL. [n. d.]. HAPTIC INTELLIGENTSIA – TIEL. Retrieved 2022-01-10 from https://tiel.ch/project/haptic-intelligentsia/

[100]

Pete Trautman. 2015. Assistive Planning in Complex, Dynamic Environments: A Probabilistic Approach. In 2015 IEEE International Conference on Systems, Man, and Cybernetics. 3072–3078. https://doi.org/10.1109/SMC.2015.534

Digital Library

[101]

E. Tunstel, M. Maimone, A. Trebi-Ollennu, J. Yen, R. Petras, and R. Willson. 2005. Mars Exploration Rover mobility and robotic arm operational performance. In 2005 IEEE International Conference on Systems, Man and Cybernetics, Vol. 2. 1807–1814 Vol. 2. https://doi.org/10.1109/ICSMC.2005.1571410 ISSN: 1062-922X.

[102]

Jurriaan van Diggelen and Matthew Johnson. 2019. Team Design Patterns. In Proceedings of the 7th International Conference on Human-Agent Interaction(HAI ’19). Association for Computing Machinery, New York, NY, USA, 118–126. https://doi.org/10.1145/3349537.3351892

Digital Library

[103]

Rodrigo Vicencio-Moreira, Regan L. Mandryk, Carl Gutwin, and Scott Bateman. 2014. The Effectiveness (or Lack Thereof) of Aim-Assist Techniques in First-Person Shooter Games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 937–946. https://doi.org/10.1145/2556288.2557308

Digital Library

[104]

Volvo. 2020. Lane Keeping Aid | Lane Keeping Aid | Driver support | XC60 2021 Early | Volvo Support. Retrieved 2022-01-10 from https://www.volvocars.com/en-ca/support/manuals/xc60/2020w17/driver-support/lane-keeping-aid/lane-keeping-aid

[105]

Huanran Wang and Xiaoping P. Liu. 2014. Adaptive Shared Control for a Novel Mobile Assistive Robot. IEEE/ASME Transactions on Mechatronics 19, 6 (Dec. 2014), 1725–1736. https://doi.org/10.1109/TMECH.2014.2299213 Conference Name: IEEE/ASME Transactions on Mechatronics.

[106]

Gesa Wiegand, Kai Holländer, Katharina Rupp, and Heinrich Hussmann. 2020. The Joy of Collaborating with Highly Automated Vehicles. In 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications(AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA, 223–232. https://doi.org/10.1145/3409120.3410643

Digital Library

[107]

Piyamate Wisanuvej, Petros Giataganas, Konrad Leibrandt, Jindong Liu, Michael Hughes, and Guang-Zhong Yang. 2017. Three-dimensional robotic-assisted endomicroscopy with a force adaptive robotic arm. In 2017 IEEE International Conference on Robotics and Automation (ICRA). 2379–2384. https://doi.org/10.1109/ICRA.2017.7989276

Digital Library

[108]

Junichi Yamaoka and Yasuaki Kakehi. 2013. dePENd: augmented handwriting system using ferromagnetism of a ballpoint pen. In Proceedings of the 26th annual ACM symposium on User interface software and technology(UIST ’13). Association for Computing Machinery, New York, NY, USA, 203–210. https://doi.org/10.1145/2501988.2502017

Digital Library

[109]

Bei Yuan, Eelke Folmer, and Frederick C. Harris. 2010. Game accessibility: a survey. Universal Access in the Information Society 10, 1 (June 2010), 81–100. https://doi.org/10.1007/s10209-010-0189-5

Digital Library

[110]

Amit Zoran and Joseph A. Paradiso. 2013. FreeD: a freehand digital sculpting tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’13). Association for Computing Machinery, New York, NY, USA, 2613–2616. https://doi.org/10.1145/2470654.2481361

Digital Library

Cited By

Peintner JEscher BDetjen HManger CRiener A(2024)How to Design Human-Vehicle Cooperation for Automated Driving: A Review of Use Cases, Concepts, and InterfacesMultimodal Technologies and Interaction10.3390/mti80300168:3(16)Online publication date: 26-Feb-2024
https://doi.org/10.3390/mti8030016
Hsieh F(2024)Applying “Two Heads Are Better Than One” Human Intelligence to Develop Self-Adaptive Algorithms for Ridesharing Recommendation SystemsElectronics10.3390/electronics1312224113:12(2241)Online publication date: 7-Jun-2024
https://doi.org/10.3390/electronics13122241
Saßmannshausen SOntika NPinatti De Carvalho ARouncefield MPipek V(2024)Amplifying Human Capabilities in Prostate Cancer Diagnosis: An Empirical Study of Current Practices and AI Potentials in RadiologyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642362(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642362
Show More Cited By

Index Terms

Two Heads Are Better Than One: A Dimension Space for Unifying Human and Artificial Intelligence in Shared Control
1. Human-centered computing

Recommendations

Joining the blunt and the pointy end of the spear: towards a common framework of joint action, human–machine cooperation, cooperative guidance and control, shared, traded and supervisory control
Abstract
To introduce this special issue of shared and cooperative control, we will look into history of tools in cooperation between humans and aim to unify the plethora of related concepts and definitions that have been proposed in recent years, such as ...
Exploring Shared Bodily Control: Designing Augmented Human Systems for Intra- and Inter-Corporeality
AHs '24: Proceedings of the Augmented Humans International Conference 2024

The human-computer interaction community has evolved from using body-sensing to body-actuating technologies, transforming the body’s role from a mere input to an input-output medium. With body-sensing, the separation between the human and the computer ...
Haptic shared control: smoothly shifting control authority?
Special Issue on Human-automation Coagency

Literature points to persistent issues in human-automation interaction, which are caused either when the human does not understand the automation or when the automation does not understand the human. Design guidelines for human-automation interaction ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

NSERC CREATE READi

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
1,389
Total Downloads

Downloads (Last 12 months)245
Downloads (Last 6 weeks)25

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Peintner JEscher BDetjen HManger CRiener A(2024)How to Design Human-Vehicle Cooperation for Automated Driving: A Review of Use Cases, Concepts, and InterfacesMultimodal Technologies and Interaction10.3390/mti80300168:3(16)Online publication date: 26-Feb-2024
https://doi.org/10.3390/mti8030016
Hsieh F(2024)Applying “Two Heads Are Better Than One” Human Intelligence to Develop Self-Adaptive Algorithms for Ridesharing Recommendation SystemsElectronics10.3390/electronics1312224113:12(2241)Online publication date: 7-Jun-2024
https://doi.org/10.3390/electronics13122241
Saßmannshausen SOntika NPinatti De Carvalho ARouncefield MPipek V(2024)Amplifying Human Capabilities in Prostate Cancer Diagnosis: An Empirical Study of Current Practices and AI Potentials in RadiologyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642362(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642362
Aronson RShort EGrollman DBroadbent EJu WSoh HWilliams T(2024)Intentional User Adaptation to Shared Control AssistanceProceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610977.3634953(4-12)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3610977.3634953
Yu YLakemond NHolmberg G(2024)Resilience in emerging complex intelligent systems: A case study of search and rescueJournal of Contingencies and Crisis Management10.1111/1468-5973.1262632:4Online publication date: 5-Oct-2024
https://doi.org/10.1111/1468-5973.12626
Shui BShi CYang YNie X(2024)ContextVis: Envision Contextual Learning and Interaction with Generative ModelsHCI International 2024 Posters10.1007/978-3-031-61953-3_24(214-224)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61953-3_24
Escarce Junior MRossmann Martins GSoriano Marcolino LRubegni E(2023)The Aesthetics of Disharmony: Harnessing Sounds and Images for Dynamic Soundscapes GenerationProceedings of the ACM on Human-Computer Interaction10.1145/36110457:CHI PLAY(665-698)Online publication date: 4-Oct-2023
https://dl.acm.org/doi/10.1145/3611045
Xu XYu AJonker TTodi KLu FQian XEvangelista Belo JWang TLi MMun AWu TShen JZhang TKokhlikyan NWang FSorenson PKim SBenko H(2023)XAIR: A Framework of Explainable AI in Augmented RealityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581500(1-30)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581500
Cimolino GChen RGutwin CGraham T(2023)Automation Confusion: A Grounded Theory of Non-Gamers’ Confusion in Partially Automated Action GamesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581116(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581116
Das ALiu HKovatchev VLease M(2023)The state of human-centered NLP technology for fact-checkingInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10321960:2Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1016/j.ipm.2022.103219

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten