skip to main content
10.1145/3663976.3663983acmotherconferencesArticle/Chapter ViewAbstractPublication PagescvipprConference Proceedingsconference-collections
research-article

A Vision-based Remote Assistance Method and it's Application in Object Transfer

Published: 27 June 2024 Publication History

Abstract

Remote assistance by users is a common means in current robot applications, which can improve the efficiency of transfer tasks. However, the current mainstream methods are costly, relying on auxiliary equipment, and result in a significant workload for users throughout the task. To reduce the cost and workload of remote assistance and improve user interaction with robots. This paper has undertaken the following work using available visual image information. Firstly, proposing a remote guidance method based on facial information, which is derived from analyzing user facial information for gaze estimation and combined with image detection. Secondly, proposing an autonomous transfer method based on position inference, which combines target information in images with the real-time state of the robot. Finally, by integrating the above methods, proposing a vision-based remote assistance method and applyingit in real-world experiments of object transfer tasks.

References

[1]
[1] Meng L, Liu J, Chai W, et al. Virtual Reality Based Robot Teleoperation via Human-Scene Interaction[J]. Procedia Computer Science, 2023, 226: 141-148.
[2]
[2] Wang J W, Sun L, Zhu X, et al. A simple approach for general task-oriented picking using placing constraints[J]. arXiv preprint arXiv:2304.01290, 2023.
[3]
[3] Chandan K, Albertson J, Zhang X, et al. Learning to guide human attention on mobile telepresence robots with 360 vision[C]//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021: 5297-5304.
[4]
[4] Lim D, Kim D, Park J. Online telemanipulation framework on humanoid for both manipulation and imitation[C]//2022 19th International Conference on Ubiquitous Robots (UR). IEEE, 2022: 8-15.
[5]
[5] Park K B, Choi S H, Lee J Y, et al. Hands-free human–robot interaction using multimodal gestures and deep learning in wearable mixed reality[J]. IEEE Access, 2021, 9: 55448-55464.
[6]
[6] Shome R, Bekris K E. Anytime multi-arm task and motion planning for pick-and-place of individual objects via handoffs[C]//2019 International Symposium on Multi-Robot and Multi-Agent Systems (MRS). IEEE, 2019: 37-43.
[7]
[7] A. Dobson, K. Solovey, R. Shome, D. Halperin, and K. E. Bekris, “Scalable Asymptotically-Optimal Multi-Robot Motion Planning,” in MRS, 2017.
[8]
[8] Zeng A, Florence P, Tompson J, et al. Transporter networks: Rearranging the visual world for robotic manipulation[C]//Conference on Robot Learning. PMLR, 2021: 726-747.
[9]
[9] Dian Wang, et al. "A Scooter-Mounted Robot Arm to Assist with Activities of Daily Life," in CoRR, vol. abs/1809.09541, 2018.
[10]
[10] Meenakshi Prabhakar et al 2021 IOP Conf. Ser.: Mater. Sci. Eng. 1012 012003
[11]
[11] Holman B, Anwar A, Singh A, et al. Watch where you’re going! gaze and head orientation as predictors for social robot navigation[C]//2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021: 3553-3559.
[12]
[12] Hart J, Mirsky R, Xiao X, et al. Using human-inspired signals to disambiguate navigational intentions[C]//International Conference on Social Robotics. Cham: Springer International Publishing, 2020: 320-331.
[13]
[13] Valenti R, Sebe N, Gevers T. Combining head pose and eye location information for gaze estimation[J]. IEEE Transactions on Image Processing, 2011, 21(2): 802-815.

Index Terms

  1. A Vision-based Remote Assistance Method and it's Application in Object Transfer

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    CVIPPR '24: Proceedings of the 2024 2nd Asia Conference on Computer Vision, Image Processing and Pattern Recognition
    April 2024
    373 pages
    ISBN:9798400716607
    DOI:10.1145/3663976
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 June 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Gaze Estimation
    2. Object Transfer
    3. Position Inference
    4. Remote Assistance
    5. Visual Processing

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • the Key Project of NSFC
    • the Science and Technology Innovation Fund of Dalian
    • the Program for Innovative Research Team in University of Liaoning Province
    • the Support Plan for Leading Innovation Team of Dalian University
    • the Support Plan for Key Field Innovation Team of Dalian
    • 111 Project
    • the Scientifc Research Funds of Education Department of Liaoning Province

    Conference

    CVIPPR 2024

    Acceptance Rates

    Overall Acceptance Rate 14 of 38 submissions, 37%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 14
      Total Downloads
    • Downloads (Last 12 months)14
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media