research-article

Towards assembly steps recognition in augmented reality

Authors:

Alia Rukubayihunga,

Jean-Yves Didier,

Samir OtmaneAuthors Info & Claims

VRIC '16: Proceedings of the 2016 Virtual Reality International Conference

Article No.: 17, Pages 1 - 5

https://doi.org/10.1145/2927929.2927953

Published: 23 March 2016 Publication History

Abstract

Augmented Reality is a media which purpose is to attach digital information to real world scenes in order to enhance the user experience. It has been used in the field of maintenance in order to show the user the operations he has to perform.

Our goal is to go one step further so that our system is able to detect when the user has performed a step of the task. It requires some understanding of what is occurring and where objects are located in order to display correct instructions for the task.

This paper is focusing on using an intermediate computation result of the usual augmented reality process, which is the pose computation: we propose to use the transformation matrix not only for objects pose estimation, but also to characterise their motion during an assembly task. With this matrix, we can induce spatial relationship between assembly parts and determine which motion occurs. Then we analyse translation and rotation parameters contained in the transformation matrix during the action. We demonstrate that these data correctly characterise the movement between object's fragments. Therefore, by analysing such a matrix, not only we can achieve the required registration step of the augmented reality process, but we can also understand the actions performed by the user.

References

[1]

H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool. Speeded-up robust features (surf). Computer Vision and Image Understanding, 110(3):346 -- 359, 2008. Similarity Matching in Computer Vision and Multimedia.

Digital Library

[2]

M. Billinghurst, M. Hakkarainen, and C. Woodward. Augmented assembly using a mobile phone. In Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia, MUM '08, pages 84--87, New York, NY, USA, 2008. ACM.

Digital Library

[3]

G. Bleser, D. Damen, A. Behera, G. Hendeby, K. Mura, M. Miezal, A. Gee, N. Petersen, G. Maçães, H. Domingues, et al. Cognitive learning, monitoring and assistance of industrial workflows using egocentric sensor networks. PloS one, 10(6), 2015.

[4]

J. Carmigniani and B. Furht. Augmented reality: an overview. In Handbook of Augmented Reality, page 3--46. Springer, 2011.

[5]

B. A. Delail, L. Weruaga, and M. J. Zemerly. Caviar: Context aware visual indoor augmented reality for a university campus. In Proceedings of the The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology - Volume 03, WI-IAT '12, pages 286--290, Washington, DC, USA, 2012. IEEE Computer Society.

Digital Library

[6]

J.-Y. Didier, D. Roussel, M. Mallem, S. Otmane, S. Naudet, Q.-C. Pham, S. Bourgeois, C. Mégard, C. Leroux, and A. Hocquard. Amra: Augmented reality assistance in train maintenance tasks. In Workshop on Industrial Augmented Reality (ISMAR'05), Vienna (Austria), October 5th 2005.

[7]

W. Friedrich, D. Jahn, and L. Schmidt. ARVIKA-Augmented reality for development, production and service. In ISMAR, volume 2002, page 3--4. Citeseer, 2002.

[8]

T. Gleue and P. Dähne. Design and implementation of a mobile device for outdoor augmented reality in the archeoguide project. In Proceedings of the 2001 Conference on Virtual Reality, Archeology, and Cultural Heritage, VAST '01, pages 161--168, New York, NY, USA, 2001. ACM.

Digital Library

[9]

V. Huikari, H. Koskimäki, P. Siirtola, and J. Roning. User-independent activity recognition for industrial assembly lines-feature vs. instance selection. In Pervasive Computing and Applications (ICPCA), 2010 5th International Conference on, pages 307--312, Dec 2010.

[10]

T. Höllerer, S. Feiner, T. Terauchi, G. Rashid, and D. Hallaway. Exploring mars: Developing indoor and outdoor user interfaces to a mobile augmented reality system. Computers and Graphics, 23:779--785, 1999.

[11]

K. Ikeuchi, M. Kawade, and T. Suehiro. Assembly task recognition with planar, curved and mechanical contacts. In Robotics and Automation, 1993. Proceedings., 1993 IEEE International Conference on, pages 688--694 vol.2, May 1993.

[12]

G. Lee and M. Billinghurst. Cityviewar outdoor ar visualization. In Proceedings of the 13th International Conference of the NZ Chapter of the ACM's Special Interest Group on Human-Computer Interaction, CHINZ '12, pages 97--97, New York, NY, USA, 2012. ACM.

Digital Library

[13]

V. Lepetit, F. Moreno-Noguer, and P. Fua. Epnp: An accurate o(n) solution to the pnp problem. International Journal of Computer Vision, 81(2):155--166, 2009.

Digital Library

[14]

M. Muja and D. G. Lowe. Fast approximate nearest neighbors with automatic algorithm configuration. In International Conference on Computer Vision Theory and Application VISSAPP'09), pages 331--340. INSTICC Press, 2009.

[15]

J. Park. Augmented reality based re-formable mock-up for design evaluation. In Ubiquitous Virtual Reality, 2008. ISUVR 2008. International Symposium on, pages 17--20, July 2008.

Digital Library

[16]

J. Sääski, T. Salonen, M. Hakkarainen, S. Siltanen, C. Woodward, and J. Lempiäinen. Integration of design and assembly using augmented reality. In Micro-Assembly Technologies and Applications, page 395--404. Springer, 2008.

[17]

B. Thomas, B. Close, J. Donoghue, J. Squires, P. De Bondi, M. Morris, and W. Piekarski. Arquake: an outdoor/indoor augmented reality first person application. In Wearable Computers, The Fourth International Symposium on, pages 139--146, Oct 2000.

[18]

J. Ward, P. Lukowicz, G. Troster, and T. Starner. Activity recognition of assembly tasks using body-worn microphones and accelerometers. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 28(10):1553--1567, Oct 2006.

Digital Library

[19]

Z. Zhang. Flexible camera calibration by viewing a plane from unknown orientations. In Computer Vision, 1999. The Proceedings of the Seventh IEEE International Conference on, volume 1, pages 666--673. IEEE, 1999.

Cited By

Bahnsen KTiemann LPlabst LGrundgeiger T(2024)Augmented Reality Cues Facilitate Task Resumption after Interruptions in Computer-Based and Physical TasksProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642666(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642666
Eversberg LGrosenick PMeusel MLambrecht J(2021)An Industrial Assistance System with Manual Assembly Step Recognition in Virtual Reality2021 International Conference on Applied Artificial Intelligence (ICAPAI)10.1109/ICAPAI49758.2021.9462061(1-6)Online publication date: 19-May-2021
https://doi.org/10.1109/ICAPAI49758.2021.9462061
Mengoni MCeccacci SGenerosi ALeopardi A(2018)Spatial Augmented Reality: an application for human work in smart manufacturing environmentProcedia Manufacturing10.1016/j.promfg.2018.10.07217(476-483)Online publication date: 2018
https://doi.org/10.1016/j.promfg.2018.10.072

Recommendations

Haptics in Augmented Reality
ICMCS '99: Proceedings of the IEEE International Conference on Multimedia Computing and Systems - Volume 2

An augmented reality system merges synthetic sensory information into a user's perception of a three-dimensional environment. An important performance goal for an augmented reality system is that the user perceives a single seamless environment. In most ...
Augmented reality as perceptual reality
VSMM'06: Proceedings of the 12th international conference on Interactive Technologies and Sociotechnical Systems

As shown in Paul Milgram et al’s Reality-Virtuality Continuum (1994), Augmented Reality occupies a very unique status in the spectrum of Mixed Reality. Unlike Virtual Reality, which is completely made up of the virtual and has been the most important ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

VRIC '16: Proceedings of the 2016 Virtual Reality International Conference

March 2016

131 pages

ISBN:9781450341806

DOI:10.1145/2927929

Conference Chair:
Simon Richir
Arts et Metiers ParisTech, France

Copyright © 2016 ACM.

© 2016 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 March 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

VRIC '16

VRIC '16: Virtual Reality International Conference - Laval Virtual 2016

March 23 - 25, 2016

Laval, France

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
142
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bahnsen KTiemann LPlabst LGrundgeiger T(2024)Augmented Reality Cues Facilitate Task Resumption after Interruptions in Computer-Based and Physical TasksProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642666(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642666
Eversberg LGrosenick PMeusel MLambrecht J(2021)An Industrial Assistance System with Manual Assembly Step Recognition in Virtual Reality2021 International Conference on Applied Artificial Intelligence (ICAPAI)10.1109/ICAPAI49758.2021.9462061(1-6)Online publication date: 19-May-2021
https://doi.org/10.1109/ICAPAI49758.2021.9462061
Mengoni MCeccacci SGenerosi ALeopardi A(2018)Spatial Augmented Reality: an application for human work in smart manufacturing environmentProcedia Manufacturing10.1016/j.promfg.2018.10.07217(476-483)Online publication date: 2018
https://doi.org/10.1016/j.promfg.2018.10.072

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten