skip to main content
10.1145/3025171.3025179acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
research-article

Adaptive View Management for Drone Teleoperation in Complex 3D Structures

Published: 07 March 2017 Publication History

Abstract

Drone navigation in complex environments poses many problems to teleoperators. Especially in 3D structures like buildings or tunnels, viewpoints are often limited to the drone's current camera view, nearby objects can be collision hazards, and frequent occlusion can hinder accurate manipulation. To address these issues, we have developed a novel interface for teleoperation that provides a user with environment-adaptive viewpoints that are automatically configured to improve safety and smooth user operation. This real-time adaptive viewpoint system takes robot position, orientation, and 3D pointcloud information into account to modify user-viewpoint to maximize visibility. Our prototype uses simultaneous localization and mapping (SLAM) based reconstruction with an omnidirectional camera and we use resulting models as well as simulations in a series of preliminary experiments testing navigation of various structures. Results suggest that automatic viewpoint generation can outperform first and third-person view interfaces for virtual teleoperators in terms of ease of control and accuracy of robot operation.

Supplementary Material

suppl.mov (iuifp0168-file3.avi)
Supplemental video

References

[1]
D. Scaramuzza, M. C. Achtelik, L. Doitsidis, F. Friedrich, E. Kosmatopoulos, A. Martinelli, M. W. Achtelik, M. Chli, S. Chatzichristo's, L. Kneip et al., "Vision-controlled micro flying robots: from system design to autonomous navigation and mapping in gps-denied environments," IEEE Robotics & Automation Magazine, vol. 21, no. 3, pp. 26--40, 2014.
[2]
D. W. Hainsworth, "Teleoperation user interfaces for mining robotics," Autonomous Robots, vol. 11, no. 1, pp. 19--28, 2001.
[3]
K. Yamazawa, Y. Yagi, and M. Yachida, "Omnidirectional imaging with hyperboloidal projection," in Intelligent Robots and Systems '93, IROS '93. Proceedings of the 1993 IEEE/RSJ International Conference on, vol. 2. IEEE, 1993, pp. 1029--1034.
[4]
L. E. Nacke, M. N. Grimshaw, and C. A. Lindley, "More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game," Interacting with Computers, vol. 22, no. 5, pp. 336--343, 2010.
[5]
S. MAEYAMA, T. OKUNO, and K. WATANABE, "View point decision algorithm for an autonomous robot to provide support images in the operability of a teleoperated robot," SICE Journal of Control, Measurement, and System Integration, vol. 9, no. 1, pp. 33--41, 2016.
[6]
N. Shiroma, N. Sato, Y.-h. Chiu, and F. Matsuno, "Study on effective camera images for mobile robot teleoperation," in Robot and Human Interactive Communication, 2004. ROMAN 2004. 13th IEEE International Workshop on. IEEE, 2004, pp. 107--112.
[7]
R. A. Manning and C. R. Dyer, "Interpolating view and scene motion by dynamic view morphing," in Computer Vision and Pattern Recognition, 1999. IEEE Computer Society Conference on., vol. 1. IEEE, 1999.
[8]
S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen, "The lumigraph," in Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 1996, pp. 43--54.
[9]
F. Okura, Y. Ueda, T. Sato, and N. Yokoya, "{paper} free-viewpoint mobile robot teleoperation interface using view-dependent geometry and texture," ITE Transactions on Media Technology and Applications, vol. 2, no. 1, pp. 82--93, 2014.
[10]
J. Engel, J. Sturm, and D. Cremers, "Camera-based navigation of a low-cost quadrocopter," in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012, pp. 2815--2821.
[11]
V. M. Monajjemi, J. Wawerla, R. Vaughan, and G. Mori, "Hri in the sky: Creating and commanding teams of uavs with a vision-mediated gestural interface," in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2013, pp. 617--623.
[12]
D. Saakes, V. Choudhary, D. Sakamoto, M. Inami, and T. Lgarashi, "A teleoperating interface for ground vehicles using autonomous flying cameras," in Artificial Reality and Telexistence (ICAT), 2013 23rd International Conference on. IEEE, 2013, pp. 13--19.
[13]
S. Pathak, A. Moro, A. Yamashita, and H. Asama, "A decoupled virtual camera using spherical optical flow," in Image Processing (ICIP), 2016 IEEE International Conference on. IEEE, 2016, pp. 4488--4492.
[14]
J. Orlosky, K. Kiyokawa, and H. Takemura, "Dynamic text management for see-through wearable and heads-up display systems," in Proceedings of the 2013 international conference on Intelligent user interfaces. ACM, 2013, pp. 363--370.
[15]
R. Grasset, T. Langlotz, D. Kalkofen, M. Tatzgern, and D. Schmalstieg, "Image-driven view management for augmented reality browsers," in Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on. IEEE, 2012, pp. 177--186.
[16]
J. Engel, T. Schöps, and D. Cremers, "Lsd-slam: Large-scale direct monocular slam," in European Conference on Computer Vision. Springer, 2014, pp. 834--849.
[17]
H. Kato, "Artoolkit: library for vision-based augmented reality," IEICE, PRMU, vol. 6, pp. 79--86, 2002.
[18]
K. Shoemake, "Animating rotation with quaternion curves," in ACM SIGGRAPH computer graphics, vol. 19, no. 3. ACM, 1985, pp. 245--254.

Cited By

View all
  • (2024)Impact of Body Movement and Teleoperation Interface on Operator's PerformanceCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640645(659-663)Online publication date: 11-Mar-2024
  • (2024)Cosmic Troubleshooting: Exploring Third-Person View for Error Handling in Telerobotic Planetary Infrastructure Maintenance2024 International Conference on Space Robotics (iSpaRo)10.1109/iSpaRo60631.2024.10687823(334-341)Online publication date: 24-Jun-2024
  • (2023)Vicarious: Context-aware Viewpoints Selection for Mixed Reality CollaborationProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615709(1-11)Online publication date: 9-Oct-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces
March 2017
654 pages
ISBN:9781450343480
DOI:10.1145/3025171
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. adaptive view
  2. drone
  3. navigation
  4. virtual reality

Qualifiers

  • Research-article

Funding Sources

Conference

IUI'17
Sponsor:

Acceptance Rates

IUI '17 Paper Acceptance Rate 63 of 272 submissions, 23%;
Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)62
  • Downloads (Last 6 weeks)4
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Impact of Body Movement and Teleoperation Interface on Operator's PerformanceCompanion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction10.1145/3610978.3640645(659-663)Online publication date: 11-Mar-2024
  • (2024)Cosmic Troubleshooting: Exploring Third-Person View for Error Handling in Telerobotic Planetary Infrastructure Maintenance2024 International Conference on Space Robotics (iSpaRo)10.1109/iSpaRo60631.2024.10687823(334-341)Online publication date: 24-Jun-2024
  • (2023)Vicarious: Context-aware Viewpoints Selection for Mixed Reality CollaborationProceedings of the 29th ACM Symposium on Virtual Reality Software and Technology10.1145/3611659.3615709(1-11)Online publication date: 9-Oct-2023
  • (2023)BirdViewAR: Surroundings-aware Remote Drone Piloting Using an Augmented Third-person PerspectiveProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580681(1-19)Online publication date: 19-Apr-2023
  • (2023)Exocentric Control Scheme for Robot Applications: An Immersive Virtual Reality ApproachIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.316038929:7(3392-3404)Online publication date: 1-Jul-2023
  • (2021)Adaptive attention-based human machine interface system for teleoperation of industrial vehicleScientific Reports10.1038/s41598-021-96682-011:1Online publication date: 26-Aug-2021
  • (2021)Drone Service Management in Emergency SituationsEmerging Technologies in Data Mining and Information Security10.1007/978-981-15-9927-9_93(983-988)Online publication date: 29-Jun-2021
  • (2021)Modeling Viewpoint of Forklift Operators Using Context-Based Clustering of Gaze FixationsHCI International 2021 - Late Breaking Papers: Multimodality, eXtended Reality, and Artificial Intelligence10.1007/978-3-030-90963-5_19(235-249)Online publication date: 11-Nov-2021
  • (2019)High-Level Teleoperation System for Aerial Exploration of Indoor EnvironmentsFrontiers in Robotics and AI10.3389/frobt.2019.000956Online publication date: 23-Oct-2019
  • (2019)Third-Person Piloting: Increasing Situational Awareness using a Spatially Coupled Second DroneProceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology10.1145/3332165.3347953(507-519)Online publication date: 17-Oct-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media