Skip to main content
SpringerLink
Log in

Position-based augmented reality platform for aiding construction and inspection of offshore plants

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

We propose an augmented reality platform to help workers with fabrication and inspection of offshore structures. The platform is designed to work under various constraints that are commonly encountered in an industrial environment and often make the existing AR methods fail to work properly. It consists of modules for tracking, registration, and augmentation. The tracking module estimates the worker’s position in real time. The registration module aligns the actual objects with 3D computer-aided design models to estimate an accurate pose of the worker’s mobile device. The augmentation module correctly augments information on the target objects on the device screen using the pose data. We test several application scenarios to demonstrate the feasibility of the proposed platform for the fabrication and inspection processes of an offshore plant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Availability of data and material (data transparency)

The data that support the findings of this study are available on request from the corresponding author, Kwanghee Ko. Part of the data are not publicly available due to some restriction, e.g., their information or data that could compromise the privacy of research participants.

Code availability (software application or custom code)

The programs developed in the work are available upon request as long as they as a whole or part of them are not used for any commercial purposes.

References

  1. Aiger, D., Mitra, N.J.: Cohen-Or D 4-points congruent sets for robust pairwise surface registration. ACM Trans. Graph. 27, 85 (2008)

    Article  Google Scholar 

  2. Bosché, F.: Plane-based registration of construction laser scans with 3D/4D building models. Adv. Eng. Inform. 26(1), 90–102 (2012)

    Article  Google Scholar 

  3. Bueno, M., Bosché, F., González-Jorge, H., Martínez-Sánchez, J., Arias, P.: 4-plane congruent sets for automatic registration of as-is 3D point clouds with 3D BIM models. Autom. Constr. 89, 120–134 (2018)

    Article  Google Scholar 

  4. Cadenam, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J.L., Reid, I., Leonard, J.J.: Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans. Robot. 32(6), 1309–1332 (2016)

    Article  Google Scholar 

  5. Corsini, M., Cignoni, P., Scopigno, R.: Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Trans. Vis. Comput. Graph. 18, 914–24 (2012). https://doi.org/10.1109/TVCG.2012.34

    Article  Google Scholar 

  6. Feigl, T., Porada, A., Steiner, S., Loffler, C., Philippsen, M.: ocalization limitations of arcore, arkit, and hololens in dynamic large-scale industry environment. In: 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Valletta, Malta (2020)

  7. Filippeschi, A., Schmitz, N., Miezal, M., Bleser, G., Ruffaldi, E., Stricker, D.: Survey of motion tracking methods based on inertial sensors: a focus on upper limb human motion. Sensors 17(6), 1257 (2017)

    Article  Google Scholar 

  8. Lee, J.M., Lee, K.H., Kim, D.S., Kim, C.H.: Active insp ection supporting system based on mixed reality after design and manufacture in an offshore structure. J. Mech. Sci. Technol. 24(1), 197–202 (2010)

    Article  Google Scholar 

  9. Leu, M.C., ElMaraghy, H.A., Nee, A.Y., Ong, S.K., Lanzetta, M., Putz, M., Zhu, W., Bernard, A.: Cad model based virtual assembly simulation, planning and training. CIRP Ann. 62(2), 799–822 (2013)

    Article  Google Scholar 

  10. Lu, C., Uchiyama, H., Thomas, D., Shimada, A., Ri, Taniguchi: Indoor positioning system based on chest-mounted imu. Sensors 19(2), 420 (2019)

    Article  Google Scholar 

  11. Madgwick, S.O.H., Harrison, A.J.L., Vaidyanathan, R.: Estimation of imu and marg orientation using a gradient descent algorithm. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–7. https://doi.org/10.1109/ICORR.2011.5975346(2011)

  12. Mellado, N., Aiger, D., Mitra, N.J.: Super 4pcs fast global pointcloud registration via smart indexing. Comput. Graph. Forum 33(5), 205–215 (2014). https://doi.org/10.1111/cgf.12446

    Article  Google Scholar 

  13. Mohssen, N., Momtaz. R., Aly, H., Youssef, M.: It’s the human that matters: accurate user orientation estimation for mobile computing applications. CoRR ArXiv:1411.2156 (2014)

  14. Morell-Gimenez, V., Saval-Calvo, M., Villena-Martinez, V., Azorin-Lopez, J., Garcia-Rodriguez, J., Cazorla, M., Orts-Escolano, S., Fuster-Guillo, A.: A survey of 3D rigid registration methods for RGB-D cameras. In: Garcia-Rodriguez, J. (ed.) Advancements in Computer Vision and Image Processing, pp. 74–98. IGI Global (2018)

  15. Mur-Artal, R., Tardos, J.: ORB-SLAM2: An open-source slam system for monocular, stereo and RGB-D cameras. IEEE Trans. Robot. (2016). https://doi.org/10.1109/TRO.2017.2705103

    Article  Google Scholar 

  16. Norrdine, A., Kasmi, Z., Blankenbach, J.: Step detection for zupt-aided inertial pedestrian navigation system using foot-mounted permanent magnet. IEEE Sens. J. 16(17), 6766–6773 (2016)

    Article  Google Scholar 

  17. Olbrich, M., Wuest, H., Riess, P., Bockholt, U.: Augmented reality pipe layout planning in the shipbuilding industry. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 269–270, IEEE (2011)

  18. Rusinkiewicz, S., Levoy, M.: Efficient variants of the icp algorithm. In: International Conference on 3-D Digital Imaging and Modeling(3DIM), pp. 145–152 (2001)

  19. Schnabel, R., Wahl, R., Klein, R.: Efficient ransac for point-cloud shape detection. Comput. Graph. Forum 26, 214–226 (2007). https://doi.org/10.1111/j.1467-8659.2007.01016.x

    Article  Google Scholar 

  20. Son, H., Bosché, F., Kim, C.: As-built data acquisition and its use in production monitoring and automated layout of civil infrastructure: a survey. Adv. Eng. Inform. 29(2), 172–183 (2015)

    Article  Google Scholar 

  21. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice, pp. 298–372. Springer, Berlin (2000)

    Chapter  Google Scholar 

  22. Wang, X., Truijens, M., Hou, L., Wang, Y., Zhou, Y.: Integrating augmented reality with building information modeling: onsite construction process controlling for liquefied natural gas industry. Autom. Constr. 40, 96–105 (2014)

    Article  Google Scholar 

  23. Yun, D., Kim, S., Heo, H., Ko, K.H.: Automated registration of multi-view point clouds using sphere targets. Adv. Eng. Inform. 29(4), 930–939 (2015)

    Article  Google Scholar 

  24. Yun, D.H., Choi, S.I., Kim, S.H., Ko, K.H.: Registration of multiview point clouds for application to ship fabrication. Graph. Models 90, 1–12 (2017)

    Article  Google Scholar 

  25. Yun, X., Bachmann, E.R., Moore, H., Calusdian, J.: Self-contained position tracking of human movement using small inertial/magnetic sensor modules. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 2526–2533. https://doi.org/10.1109/ROBOT.2007.363845(2007)

  26. Zollmann, S., Hoppe, C., Kluckner, S., Poglitsch, C., Bischof, H., Reitmayr, G.: Augmented reality for construction site monitoring and documentation. Proc. IEEE 102(2), 137–154 (2014)

    Article  Google Scholar 

Download references

Funding

This work was supported by the National IT Industry Promotion Agency (NIPA), grant funded by the Korean Government Ministry of Science and ICT (MSIT). Grant No. S0602-17-1021, for development of a smart mixed reality technology for improving the pipe installation and inspection processes in the offshore structure fabrication.

Author information

Authors and Affiliations

  1. The School of Mechanical Engineering, Gwangju Institute of Science and Technology, 123 Cheomdangwagiro, Gwangju, 61005, Republic of Korea

    Junho Choi, Moon Gu Son, Yong Yi Lee, Kwan H. Lee & Kwanghee Ko

  2. SOFTHILLS Co., Ltd., Seoul, 04780, Republic of Korea

    Jin Pyo Park & Chang Hun Yeo

  3. SAMSUNG Heavy Industries, Geoju, 53261, Republic of Korea

    Jungseo Park & Sungin Choi

  4. Marine Tech-In Co., Ltd., Busan, 48059, Republic of Korea

    Won Don Kim & Tae Won Kang

Authors
  1. Junho Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Moon Gu Son
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Yi Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kwan H. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jin Pyo Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chang Hun Yeo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jungseo Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sungin Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Won Don Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Tae Won Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Kwanghee Ko
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception, design, development, implementation and experiments. Material preparation, data collection, implementation, experiments, and analysis were performed by JC, MGS, YYL, KHL, JPP, CHY, JP, SC, WDK, TWK and KK. The first draft of the manuscript was written by JC, MGS and KK, and all authors commented on the previous version of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Kwanghee Ko.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (avi 50734 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, J., Son, M.G., Lee, Y.Y. et al. Position-based augmented reality platform for aiding construction and inspection of offshore plants. Vis Comput 36, 2039–2049 (2020). https://doi.org/10.1007/s00371-020-01902-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-020-01902-9

Keywords

Search

Navigation