Abstract
We propose an augmented reality platform to help workers with fabrication and inspection of offshore structures. The platform is designed to work under various constraints that are commonly encountered in an industrial environment and often make the existing AR methods fail to work properly. It consists of modules for tracking, registration, and augmentation. The tracking module estimates the worker’s position in real time. The registration module aligns the actual objects with 3D computer-aided design models to estimate an accurate pose of the worker’s mobile device. The augmentation module correctly augments information on the target objects on the device screen using the pose data. We test several application scenarios to demonstrate the feasibility of the proposed platform for the fabrication and inspection processes of an offshore plant.











Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and material (data transparency)
The data that support the findings of this study are available on request from the corresponding author, Kwanghee Ko. Part of the data are not publicly available due to some restriction, e.g., their information or data that could compromise the privacy of research participants.
Code availability (software application or custom code)
The programs developed in the work are available upon request as long as they as a whole or part of them are not used for any commercial purposes.
References
Aiger, D., Mitra, N.J.: Cohen-Or D 4-points congruent sets for robust pairwise surface registration. ACM Trans. Graph. 27, 85 (2008)
Bosché, F.: Plane-based registration of construction laser scans with 3D/4D building models. Adv. Eng. Inform. 26(1), 90–102 (2012)
Bueno, M., Bosché, F., González-Jorge, H., Martínez-Sánchez, J., Arias, P.: 4-plane congruent sets for automatic registration of as-is 3D point clouds with 3D BIM models. Autom. Constr. 89, 120–134 (2018)
Cadenam, C., Carlone, L., Carrillo, H., Latif, Y., Scaramuzza, D., Neira, J.L., Reid, I., Leonard, J.J.: Past, present, and future of simultaneous localization and mapping: toward the robust-perception age. IEEE Trans. Robot. 32(6), 1309–1332 (2016)
Corsini, M., Cignoni, P., Scopigno, R.: Efficient and flexible sampling with blue noise properties of triangular meshes. IEEE Trans. Vis. Comput. Graph. 18, 914–24 (2012). https://doi.org/10.1109/TVCG.2012.34
Feigl, T., Porada, A., Steiner, S., Loffler, C., Philippsen, M.: ocalization limitations of arcore, arkit, and hololens in dynamic large-scale industry environment. In: 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Valletta, Malta (2020)
Filippeschi, A., Schmitz, N., Miezal, M., Bleser, G., Ruffaldi, E., Stricker, D.: Survey of motion tracking methods based on inertial sensors: a focus on upper limb human motion. Sensors 17(6), 1257 (2017)
Lee, J.M., Lee, K.H., Kim, D.S., Kim, C.H.: Active insp ection supporting system based on mixed reality after design and manufacture in an offshore structure. J. Mech. Sci. Technol. 24(1), 197–202 (2010)
Leu, M.C., ElMaraghy, H.A., Nee, A.Y., Ong, S.K., Lanzetta, M., Putz, M., Zhu, W., Bernard, A.: Cad model based virtual assembly simulation, planning and training. CIRP Ann. 62(2), 799–822 (2013)
Lu, C., Uchiyama, H., Thomas, D., Shimada, A., Ri, Taniguchi: Indoor positioning system based on chest-mounted imu. Sensors 19(2), 420 (2019)
Madgwick, S.O.H., Harrison, A.J.L., Vaidyanathan, R.: Estimation of imu and marg orientation using a gradient descent algorithm. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–7. https://doi.org/10.1109/ICORR.2011.5975346(2011)
Mellado, N., Aiger, D., Mitra, N.J.: Super 4pcs fast global pointcloud registration via smart indexing. Comput. Graph. Forum 33(5), 205–215 (2014). https://doi.org/10.1111/cgf.12446
Mohssen, N., Momtaz. R., Aly, H., Youssef, M.: It’s the human that matters: accurate user orientation estimation for mobile computing applications. CoRR ArXiv:1411.2156 (2014)
Morell-Gimenez, V., Saval-Calvo, M., Villena-Martinez, V., Azorin-Lopez, J., Garcia-Rodriguez, J., Cazorla, M., Orts-Escolano, S., Fuster-Guillo, A.: A survey of 3D rigid registration methods for RGB-D cameras. In: Garcia-Rodriguez, J. (ed.) Advancements in Computer Vision and Image Processing, pp. 74–98. IGI Global (2018)
Mur-Artal, R., Tardos, J.: ORB-SLAM2: An open-source slam system for monocular, stereo and RGB-D cameras. IEEE Trans. Robot. (2016). https://doi.org/10.1109/TRO.2017.2705103
Norrdine, A., Kasmi, Z., Blankenbach, J.: Step detection for zupt-aided inertial pedestrian navigation system using foot-mounted permanent magnet. IEEE Sens. J. 16(17), 6766–6773 (2016)
Olbrich, M., Wuest, H., Riess, P., Bockholt, U.: Augmented reality pipe layout planning in the shipbuilding industry. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality, pp. 269–270, IEEE (2011)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the icp algorithm. In: International Conference on 3-D Digital Imaging and Modeling(3DIM), pp. 145–152 (2001)
Schnabel, R., Wahl, R., Klein, R.: Efficient ransac for point-cloud shape detection. Comput. Graph. Forum 26, 214–226 (2007). https://doi.org/10.1111/j.1467-8659.2007.01016.x
Son, H., Bosché, F., Kim, C.: As-built data acquisition and its use in production monitoring and automated layout of civil infrastructure: a survey. Adv. Eng. Inform. 29(2), 172–183 (2015)
Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) Vision Algorithms: Theory and Practice, pp. 298–372. Springer, Berlin (2000)
Wang, X., Truijens, M., Hou, L., Wang, Y., Zhou, Y.: Integrating augmented reality with building information modeling: onsite construction process controlling for liquefied natural gas industry. Autom. Constr. 40, 96–105 (2014)
Yun, D., Kim, S., Heo, H., Ko, K.H.: Automated registration of multi-view point clouds using sphere targets. Adv. Eng. Inform. 29(4), 930–939 (2015)
Yun, D.H., Choi, S.I., Kim, S.H., Ko, K.H.: Registration of multiview point clouds for application to ship fabrication. Graph. Models 90, 1–12 (2017)
Yun, X., Bachmann, E.R., Moore, H., Calusdian, J.: Self-contained position tracking of human movement using small inertial/magnetic sensor modules. In: Proceedings 2007 IEEE International Conference on Robotics and Automation, pp. 2526–2533. https://doi.org/10.1109/ROBOT.2007.363845(2007)
Zollmann, S., Hoppe, C., Kluckner, S., Poglitsch, C., Bischof, H., Reitmayr, G.: Augmented reality for construction site monitoring and documentation. Proc. IEEE 102(2), 137–154 (2014)
Funding
This work was supported by the National IT Industry Promotion Agency (NIPA), grant funded by the Korean Government Ministry of Science and ICT (MSIT). Grant No. S0602-17-1021, for development of a smart mixed reality technology for improving the pipe installation and inspection processes in the offshore structure fabrication.
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception, design, development, implementation and experiments. Material preparation, data collection, implementation, experiments, and analysis were performed by JC, MGS, YYL, KHL, JPP, CHY, JP, SC, WDK, TWK and KK. The first draft of the manuscript was written by JC, MGS and KK, and all authors commented on the previous version of the manuscript. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (avi 50734 KB)
Rights and permissions
About this article
Cite this article
Choi, J., Son, M.G., Lee, Y.Y. et al. Position-based augmented reality platform for aiding construction and inspection of offshore plants. Vis Comput 36, 2039–2049 (2020). https://doi.org/10.1007/s00371-020-01902-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-020-01902-9