skip to main content
10.1145/3177102.3177112acmconferencesArticle/Chapter ViewAbstractPublication PageshotmobileConference Proceedingsconference-collections
research-article

Unsupervised Workflow Extraction from First-Person Video of Mechanical Assembly

Published: 12 February 2018 Publication History

Abstract

Recently, Augmented Reality (AR) applications have proved to help improve the efficiency in accomplishing assembly tasks. However, due to the lack of approaches to automatic workflow extraction, the existing AR-based assembly assistance applications require manual authoring, which hampers scalability. Moreover, most of these applications only support information visualization and video documentation. To tackle the challenge of scalability and to enable more intelligent functionalities, such as real-time quality control, we propose in this paper a novel solution for unsupervised workflow extraction from first-person video of mechanical assembly, without any pre-labeled training data or pre-trained classifiers. Our proposed system automatically discovers a sequence of operations from the input video, and describes the extracted workflow process with semantics. Preliminary evaluation demonstrates the feasibility of our solution and highlights the technical challenges.

References

[1]
Hape International AG 2017 (accessed Oct 1, 2017). Hape kid toys company. deftempurl%https://www.hape.comshowURL%tempurl
[2]
Susanna Aromaa et al. 2016. Use of Wearable and Augmented Reality Technologies in Industrial Maintenance Work Proceedings of the 20th International Academic Mindtrek Conference (AcademicMindtrek '16). ACM, New York, NY, USA, 235--242.
[3]
Sugata Banerji et al. 2012. LBP and Color Descriptors for Image Classification. (2012).
[4]
Alejandro Betancourt et al. 2014. A sequential classifier for hand detection in the framework of egocentric vision Proceedings of the IEEE CVPR Workshops. 586--591.
[5]
Alejandro Betancourt et al. 2015. A dynamic approach and a new dataset for hand-detection in first person vision CAIP 2015. Springer, 274--287.
[6]
Alejandro Betancourt et al. 2015. Filtering SVM frame-by-frame binary classification in a detection framework Proceedings of the IEEE ICIP 2015. IEEE, 2552--2556.
[7]
Alejandro Betancourt et al. 2015. Towards a unified framework for hand-based methods in first person vision Proceedings of 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). IEEE, 1--6.
[8]
Diana Borrego et al. 2013. Diagnosing correctness of semantic workflow models. Data & Knowledge Engineering Vol. 87 (2013), 167--184.
[9]
Zhuo Chen et al. 2015. Early implementation experience with wearable cognitive assistance applications Proceedings of the 2015 workshop on Wearable Systems and Applications. ACM, 33--38.
[10]
John Ahmet Erkoyuncu et al. 2017. Improving efficiency of industrial maintenance with context aware adaptive authoring in augmented reality. CIRP Annals Vol. 66, 1 (2017), 465 -- 468. Kamala K. Mahapatra 2017. MIL based visual object tracking with kernel and scale adaptation. Signal Processing: Image Communication Vol. 53 (2017), 51--64.
[11]
X Wang et al. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing Vol. 4, 1 (2016), 1--22.
[12]
C Lawrence Zitnick and Piotr Dollár 2014. Edge boxes: Locating object proposals from edges. In ECCV. Springer, 391--405.

Cited By

View all
  • (2024)Interoperability Between AR and CAD Systems for Industrial ApplicationsDesign Tools and Methods in Industrial Engineering III10.1007/978-3-031-52075-4_63(557-564)Online publication date: 7-Feb-2024
  • (2021)Minimal AR: visual asset optimization for the authoring of augmented reality work instructions in manufacturingThe International Journal of Advanced Manufacturing Technology10.1007/s00170-021-08449-6Online publication date: 30-Nov-2021
  • (2021)Ajalon: Simplifying the authoring of wearable cognitive assistantsSoftware: Practice and Experience10.1002/spe.298751:8(1773-1797)Online publication date: 18-May-2021

Index Terms

  1. Unsupervised Workflow Extraction from First-Person Video of Mechanical Assembly

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    HotMobile '18: Proceedings of the 19th International Workshop on Mobile Computing Systems & Applications
    February 2018
    130 pages
    ISBN:9781450356305
    DOI:10.1145/3177102
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 February 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. video analytics
    2. workflow extraction

    Qualifiers

    • Research-article

    Conference

    HotMobile '18
    Sponsor:

    Acceptance Rates

    HotMobile '18 Paper Acceptance Rate 19 of 65 submissions, 29%;
    Overall Acceptance Rate 96 of 345 submissions, 28%

    Upcoming Conference

    HOTMOBILE '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)19
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 27 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Interoperability Between AR and CAD Systems for Industrial ApplicationsDesign Tools and Methods in Industrial Engineering III10.1007/978-3-031-52075-4_63(557-564)Online publication date: 7-Feb-2024
    • (2021)Minimal AR: visual asset optimization for the authoring of augmented reality work instructions in manufacturingThe International Journal of Advanced Manufacturing Technology10.1007/s00170-021-08449-6Online publication date: 30-Nov-2021
    • (2021)Ajalon: Simplifying the authoring of wearable cognitive assistantsSoftware: Practice and Experience10.1002/spe.298751:8(1773-1797)Online publication date: 18-May-2021

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    EPUB

    View this article in ePub.

    ePub

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media