Skip to main content
SpringerLink
Log in

Inserting and tracking a plane object in a three-dimensional scene

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This article introduces the basic element to build an augmented reality system allowing to insert a 2D object in a real 3D scene in real time. The first step consists in locating the place where to insert the object using an abstract marker, this marker is a rectangle that surrounds the minimum area of the object's contour detected through its color. This rectangle is a plane surface that provides the position of its four points in the images acquired in real time which allows to have a real time tracking of the detected object. Planar homography describes exactly the relationship between the key points if the scene is flat and only requires four key points to produce an exact solution of the camera position to align a 2D virtual object in a 3D real scene.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Abidi MA, Chandra T (1995) A new efficient and direct solution for pose estimation using quadrangular targets: algorithm and evaluation. IEEE Trans Pattern Anal Mach Intell 17(5):534−538. https://doi.org/10.1109/34.391388

  2. Agarwal A, Jawahar CV, Narayanan PJ (2005) A survey of planar homography estimation techniques

  3. Ajanki A et al (2011) An augmented reality interface to contextual information. Virtual Real 15(2−3):161−173. https://doi.org/10.1007/s10055-010-0183-5

  4. Almenara JC, Osuna JB (2016) The educational possibilities of augmented reality. J New Approach Educ Res 6(1):44−50. https://doi.org/10.7821/naer.2016.1.140

  5. Amin D, Govilkar S (2015) Comparative Study of Augmented Reality Sdk’s. Int J Comput Sci Appl 5(1):11−26. https://doi.org/10.5121/ijcsa.2015.5102

  6. Annich A, El Abderrahmani A, Satori K (2017) Fast and easy 3D reconstruction with the help of geometric constraints and genetic algorithms. 3D Res 8(3). https://doi.org/10.1007/s13319-017-0139-6

  7. Azuma RT (1997) A survey of augmented reality. Presence: Teleoperators and Virtual Environments. p. 355-385

  8. Azuma R, Baillot Y, Behringer R, Feiner S (2011) S. Julie r and B. MacIntyre. Recent advances in augmented reality. IEEE Comput Graph Appl 21(6):34−47. https://doi.org/10.1109/38.963459

  9. Basori AH, Afif FN, Almazyad AS, AbuJabal HAS, Rehman A, Alkawaz MH (2015) Fast markerless tracking for augmented reality in planar environment. 3D Res 6(4) https://doi.org/10.1007/s13319-015-0072-5

  10. Bay H, Tuytelaars T, Van Gool L (2006) SURF: Speeded Up Robust Features. In Computer Vision – ECCV, (2006) vol. 3951, A. Leonardis, H. Bischof, et A. Pinz, Éd. Berlin, Heidelberg: Springer, Berlin Heidelberg 2006:404–417

    Google Scholar 

  11. Benhimane S, Malis E (2007) Homography-based 2D visual tracking and serving. Int J Robot Res 26(7):661−676. https://doi.org/10.1177/0278364907080252

  12. Chalhoub J, Ayer JK (2019) Exploring the performance of an augmented reality application for construction layout tasks. Multimed Tools Appl 78(24):35075−35098. https://doi.org/10.1007/s11042-019-08063-5

  13. Caudell T, Mizell D (1992) Augmented reality: An application of heads-up display technology to manual manufacturing processes. In Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences 2:659−669. https://doi.org/10.1109/HICSS.1992.183317

  14. El Batteoui I, Saaidi A, Satori K (2015) Accurate self-calibration of camera with variable Intrinsic parameters from unknown 3D scene. 3D Res 6(3). https://doi.org/10.1007/s13319-015-0065-4

  15. El Hazzat S, Saaidi A, Karam A, Satori K (2015) Incremental multi-view 3D reconstruction starting from two images taken by a stereo pair of cameras. 3D Res 6(1). https://doi.org/10.1007/s13319-015-0041-z

  16. Gauglitz S, Höllerer T, Turk M (2011) Evaluation of interest point detectors and feature descriptors for visual tracking. Int J Comput Vis 94(3):335−360. https://doi.org/10.1007/s11263-011-0431-5

  17. Geng Y et al (2017) Learning convolutional neural network to maximize Pos@Top performance measure. ArXiv160908417 Cs, [En ligne]. Disponible sur. http://arxiv.org/abs/1609.08417

  18. Golub GH, Reinsch C (1970) Singular value decomposition and least squares solutions. Numer Math 14(5):403−420. https://doi.org/10.1007/BF02163027

  19. Grasset R, Gascuel JD (2001) Environnement de Réalité Augmentée Collaboratif: Manipulation d’Objets Réels et Virtuels

  20. Gribaudo M, Moos S, Piazzolla P, Porpiglia F, Vezzetti E, Violante MG (2020) Enhancing Spatial Navigation in Robot-Assisted Surgery: An Application. In Design Tools and Methods in Industrial Engineering, Cham 95−105. https://doi.org/10.1007/978-3-030-31154-4_9

  21. Hartley R, Zisserman A (2001) Multiple view geometry in computer vision. Cambridge University Press, London, first edition

  22. Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico 1106−1112. https://doi.org/10.1109/CVPR.1997.609468

  23. Huang Z, Boufama B (2002) A semi-automatic camera calibration method for augmented reality. In IEEE International Conference on Systems, Man and Cybernetics, Yasmine Hammamet, Tunisia 4:6. https://doi.org/10.1109/ICSMC.2002.1173372

  24. Hutchison D (2010) BRIEF: Binary Robust Independent Elementary Features. In Computer Vision – ECCV, et al (2010) vol. 6314, K. Daniilidis, P. Maragos, et N. Paragios, Éd. Berlin, Heidelberg: Springer, Berlin Heidelberg 2010:778–792

    Google Scholar 

  25. Jain AK (1989) Fundamentals of digital image processing. Prentice Hall, Englewood Cliffs, NJ

    MATH  Google Scholar 

  26. Kato H, Billinghurst M (2019) Marker tracking and HMD calibration for a video-based augmented reality conferencing system 85−94. https://doi.org/10.1109/IWAR.1999.803809

  27. Kenngott HG et al (2018) Mobile, real-time, and point-of-care augmented reality is robust, accurate, and feasible: a prospective pilot study. Surg Endosc 32(6):2958−2967. https://doi.org/10.1007/s00464-018-6151-y

  28. Kim TJ, Huh JH, Kim JM (2018) Bi-directional education contents using VR equipments and augmented reality. Multimed Tools Appl 77(22):30089−30104. https://doi.org/10.1007/s11042-018-6181-4

  29. Koller D, Klinker G, Rose E, Breen D, Whitaker R, Tuceryan M (1997) Real-time vision-based camera tracking for augmented reality applications 87−94. https://doi.org/10.1145/261135.261152

  30. Lepetit V, Moreno-Noguer F, Fua P (2009) EPnP: An accurate on solution to the PnP problem. Int J Comput Vis 81(2):155−166. https://doi.org/10.1007/s11263-008-0152-6

  31. Lima JP et al (2017) Markerless tracking system for augmented reality in the automotive industry. Expert Syst Appl 82:100−114. https://doi.org/10.1016/j.eswa.2017.03.060

  32. Liu L, Li H, Gruteser M (2019) Edge assisted real-time object detection for mobile augmented reality. In The 25th Annual International Conference on Mobile Computing and Networking, Los Cabos Mexico 1−16. https://doi.org/10.1145/3300061.3300116

  33. Lowe DG (1999) Object recognition from local scale-invariant features. In Proceedings of the Seventh IEEE International Conference on Computer Vision 2:1150−1157. https://doi.org/10.1109/ICCV.1999.790410

  34. Mackay WE (1998) Augmented Reality: Linking real and virtual worlds: a new paradigm for interacting with computers.

  35. Malik S, Roth G, McDonald C (2002) Robust 2D Tracking for Real-time Augmented Reality. In: Proceedings of Vision Interface, Calgary, Alberta, Canada

  36. Marchand E, Uchiyama H, Spindler F (2016) Pose Estimation for Augmented Reality: A Hands-On Survey. IEEE Trans Vis Comput Graph 22(12):2633−2651. https://doi.org/10.1109/TVCG.2015.2513408

  37. Mooser J, Wang L, You S, Neumann U (2007) An Augmented Reality Interface for Mobile Information Retrieval. In Multimedia and Expo, 2007 IEEE International Conference on, Beijing, China 2226−2229. https://doi.org/10.1109/ICME.2007.4285128

  38. Nagymáté G, Kiss RM (2019) Affordable gait analysis using augmented reality markers. PLoS One 14(2). https://doi.org/10.1371/journal.pone.0212319

  39. Nóbrega R, Correia, N (2017) Interactive 3D content insertion in images for multimedia applications. Multimed Tools Appl 76(1):163−197. https://doi.org/10.1007/s11042-015-3031-5

  40. Oufqir Z, Abderrahmani AE, Satori K (2018) Comparative study of object insertion methods on image sequence for augmented reality 5

  41. Oufqir Z, Abderrahmani AE, Satori K (2019) Important method for detecting and tracking based on color 8(5):5

  42. Oufqir Z, El Abderrahmani A, Satori K (2020) From marker to markerless in augmented reality. In Embedded Systems and Artificial Intelligence, Singapore 599−612. https://doi.org/10.1007/978-981-15-0947-6_57

  43. Prince SJD, Xu K, Cheok AD (2002) Augmented reality camera tracking with homographies. IEEE Comput Graph Appl 22(6):39−45. https://doi.org/10.1109/MCG.2002.1046627

  44. Rambach J, Pagani A, Schneider M, Artemenko O, Stricker D (2018) 6DoF object tracking based on 3D scans for sugmented reality remote Live support. Computers 7(1): 6. https://doi.org/10.3390/computers7010006

  45. Rusiñol M, Chazalon J, Diaz-Chito K (2018) Augmented songbook: an augmented reality educational application for raising music awareness. Multimed Tools Appl 77(11):13773−13798. https://doi.org/10.1007/s11042-017-4991-4

  46. Simon G, Fitzgibbon AW, Zisserman A (2000) Markerless tracking using planar structures in the scene. In Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR (2000) Munich. Germany 120–128. https://doi.org/10.1109/ISAR.2000.880935

    Article  Google Scholar 

  47. Simon G, Berger MO (2002) Pose estimation for planar structures. IEEE Comput Graph Appl 22(6):46−53. https://doi.org/10.1109/MCG.2002.1046628

  48. Shoaib H, Jaffry SW A survey of augmented reality, International Conference on Virtual and Augmented Reality (ICV AR 2015)

  49. Tarko J, Tompkin J, Richardt C (2019) Real-time Virtual Object Insertion for Moving 360° Videos. In The 17th International Conference on Virtual-Reality Continuum and its Applications in Industry, Brisbane QLD Australia 1−9. https://doi.org/10.1145/3359997.3365708

  50. Tobias HH, Steven KF (2004) Mobile augmented reality. In Location Based Services and TeleCartography

  51. Uchiyama H, Saito H, Servières M, Moreau G (2011) Camera tracking by online learning of keypoint arrangements using LLAH in augmented reality applications. Virtual Real 15(2−3):109−117. https://doi.org/10.1007/s10055-010-0173-7

  52. Villegas-Hernandez YS, Guedea-Elizalde F (2017) Marker’s position estimation under uncontrolled environment for augmented reality. Int J Interact Des Manuf IJIDeM 11(3):727−735. https://doi.org/10.1007/s12008-016-0356-x

  53. Wedyan M, AL-Jumaily A, Dorgham O (2020) The use of augmented reality in the diagnosis and treatment of autistic children: a review and a new system. Multimed Tools Appl. https://doi.org/10.1007/s11042-020-08647-6

  54. Zhang G et al (2017) Learning Convolutional Ranking-Score Function by Query Preference Regularization. In Intelligent Data Engineering and Automated Learning – IDEAL. Cham 2017:1–8. https://doi.org/10.1007/978-3-319-68935-7_1

    Article  Google Scholar 

  55. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22:(11):1330−1334. https://doi.org/10.1109/34.888718

  56. Zhang W, Han B, Hui P (2018a) Jaguar: Low Latency Mobile Augmented Reality with Flexible Tracking. In 2018 ACM Multimedia Conference on Multimedia Conference - MM ’18, Seoul, Republic of Korea 355−363. https://doi.org/10.1145/3240508.3240561

  57. Zhang G, Liang G, Su F, Qu F, Wang JY (2018b) Cross-Domain Attribute Representation Based on Convolutional Neural Network. In Intelligent Computing Methodologies, vol. 10956, D.-S. Huang, M. M. Gromiha, K. Han, et A. Hussain, Éd. Cham: Springer International Publishing 134−142

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and informatic USMBA Faculty of Sciences Fes, LIIAN, Fes, Morocco

    Zainab Oufqir, Abdellatif EL Abderrahmani & Khalid Satori

Authors
  1. Zainab Oufqir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdellatif EL Abderrahmani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Khalid Satori
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zainab Oufqir.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Oufqir, Z., Abderrahmani, A.E. & Satori, K. Inserting and tracking a plane object in a three-dimensional scene. Multimed Tools Appl 81, 1357–1373 (2022). https://doi.org/10.1007/s11042-021-11536-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11536-1

Keywords

Search

Navigation