ARGIS-based outdoor underground pipeline information system

https://doi.org/10.1016/j.jvcir.2016.07.011Get rights and content

Highlights

  • This paper tackles the key issues and summarizes the key technologies of ARGIS.

  • This paper designs the ARGIS based underground pipeline information system.

  • Two versions (computer vision, sensor) of the system are realized.

  • This paper analyzes the user experience on CV-version, Sensor-version and PC-version.

Abstract

Outdoor augmented reality geographic information system (ARGIS) is a hot topic of augmented reality application over recent years. This paper tackles the key issues of ARGIS, designs the mobile augmented reality based underground pipeline information system, and respectively realizes the computer vision based version (CV-version) and the sensor based version (Sensor-version). The CV-version system employs the neural network based 3D features matching method. After comparison and evaluation of the two versions as well as interviewing the domain exports’ opinions on the proposed system and the traditional PC-version virtual reality pipeline information system, it indicates that the CV-version is better in virtual-reality fusion and interactive experience and that Sensor-version is better in environmental robustness; PC-version virtual reality pipeline information system is more macroscopic in demonstration communication and this system is more assistant in the site survey of urban underground pipelines. To sum up, taking the underground pipeline information system as an example, this paper studies the characteristics of two different techniques of computer vision and sensor fusion in ARGIS, and makes a detailed comparison of these two techniques from two aspects of performance parameters and user experience. Some guiding conclusions are drawn, which provides a new way for the research of the application system in the field of outdoor geographic information system.

Introduction

Since the human activities are related with the geographic information all the time, it is very natural and extremely important to integrate the augmented reality technology with the geographic information system (GIS). Since its generation, the augmented reality has gone through several development phases: computer indoor augmented reality, helmet-type outdoor augmented reality and mobile based outdoor augmented reality. Its application is gradually converted from the traditional exhibition to the application combined with the industry. This paper studies the mobile based outdoor augmented reality geographic information system (ARGIS) and analyzes it by combining the examples for underground pipeline prospect. In the process of the implementation of the examples, this paper also studies the neural network based 3D features matching method to optimize the feature matching process.

As the product of continuous development of virtual reality technology, the augmented reality can be traced back to the HMD (Head Mounted Display) invented in 1965 [1]; the user can see the superposition of real environment and 3D image. The concept of augmented reality is brought forward for the first time by Caudell and Mizell [2], researchers of Boeing Company in the early stage of 1990s. Later, the handheld device became smaller but the calculated performance was stronger. Feiner [3] designed the first prototype of mobile augmented reality system (MARS) in 1997. The system was able to add 3D travel guidance information to the real building environment. At the end of 1990s, the augmented reality became an independent and significant research field.

In recent years, with the rapid development of mobile devices represented by smart phones, its high-resolution display screen, high pixel built-in camera and strong calculating ability can well adapt to the requirements of carrying augmented reality applications. In September 2009, Apple Inc. released iOS3.1 and showed that the support of augmented reality technology was added in smart phones for the first time. Meanwhile, with its APP’s universality for mobile phones, google glass, google cardboard and all kinds of devices, the Android system popularized the augmented reality technology better.

The augmented reality mobile phone applications of Location Based Services (LBS) are the hot applications of mobile augmented reality. The iButterfly [29], LayarReality Browser [30], YelpiPhone [31], Sekai Camera [32] and NearestTube [33] etc. obtain their spatial locations through a handheld device’s GPS or gyroscope positioning and acquire relevant information via mobile network. These mobile phone APPs with augmented reality do not rely on the function of image recognition, reduce the requirements on mobile phone computation capacity but demand high communication capacity.

The augmented reality application is increasingly integrated with the industry applications, meanwhile, the integration with the GIS application is the current hotspot.

The traditional GIS application is displayed on the map in the form of 2D symbols after analyzing the result based on 2D map. However, due to the lack of strong intuitive perception, it is difficult for the user with weak geographic knowledge to distinguish it; even the field experts also have some difficulties in distinguishing in a complicated environment. For recent years, with the 3DGIS and VRGIS development, the 3D virtual model of real environment was established to replace the 2D map [4], which improves the intuition of marker distinguishing and promotes the visualization development of GIS. However, the 3D virtual model has high requirements on hardware while it still has difference with the real world scene. As a matter of fact, the real world scene is a complicated and perfect “3D space map” for itself. If the real world scene is taken as the “map” for GIS application operation and the inquired and analyzed result is displayed on the real world scene via virtual information to achieve the interaction between the virtual space geographic information and the real world scene, so the GIS application experience can be enhanced.

Carry out scene augmentation [5], [6] for human’s visual system by organically integrating the 2D or 3D pictures, text notes and other virtual information generated by the computer into the real world scene which can be seen by user. The augmented reality has the effect of expressing the sense of reality but cannot store and control data. However, GIS has the functions of storage, management and analysis of space information, which precisely makes up for this deficiency. The integration of them can provide the user with the location-based service and can combine all kinds of virtual space information stored in the GIS space data with the scene actually observed [7]. Besides, such integration can not only enhance the GIS user’s sense of real environment and the interactive experience but also supplies a new means for the research on GIS visualization.

Sun et al. [8] (2004) brought forward the concept of ARGIS, a geographic information system, digitally describes, stores and controls the objective geographic world, meanwhile, integrates such descriptions into the real world, offers the space information of a designated object and supplies the outdoor mobile information interaction. Guo et al. (2008) pointed out that the significance of ARGIS is to apply the mobile computing and augmented reality technologies to the traditional space information service to change the traditional location-based service mechanism; then, human, which is the subject, the real world, which is the object, and the digital world transmitted by internet can combined with each other seamlessly so as to realize the interaction without being limited by any time and space and to alter the human-to-digital world and human-to-real world interactive pattern, which will provide the space-based industry system with a brand-new industrial pattern [9].

Gethin D’s article [10] of 2002 adapted GPS/INS for 3D registration of outdoor geographic information, realizing the visual presentation of outdoor underground facility structure. The subsurface data visualization system of University of Nottingham, which uses the GPS/INS integrated system in system registration, can carry out the 3D visualization for all kinds of the following subsurface characteristics such as geologic structure, underground pipe system, underground land pollution zone etc. Gerhard Schall has published 15 articles on outdoor augmented reality since 2008, emphasizing the application of hardware sensor. One of these articles written in 2009 [11] designs an underground facility visualization augmented reality system frame and an integrated-form handheld device and integrates GPS, camera, wireless network and other devices but is not the mobile-phone-based application. The 3D registration uses the pure hardware sensor technology (GPS+ inertia measurement equipment). Gerhard Schall started to research the visual and sensor hybrid tracking registration method [12] in 2010 and obtained a series of research achievements [13], [14]. The article [16] designs an industrial solution, using GPS positioning and CAD pipeline data in the real environment; it highly depends on the accuracy of GPS and there is no explanations about the orientation, tracking and other technical details.

Section snippets

Spatial data organization and distribution

There are many mature schemes for the organization and distribution of spatial data at present and the distribution of OGC-based spatial data network is the present mainstream framework. OGC puts forward a frame, namely OWS (OGC Web Service), which can seamlessly integrate all kinds of online spatial processing and location service to let the distributed spatial processing system to interact through XML and HTTP technologies and to offer interoperation frame for all the online spatial data

System

Based on ARGIS framework, this paper proposed a mobile augmented reality application system for urban underground pipeline prospecting. The system takes the underground pipeline data in Qingdao (500 square kilometers of construction area and 10 million population) as the experimental data to have implemented the computer vision based version (CV-version) and sensor based version (Sensor-version), and the user experience test is carried out. The framework of the system is described as shown in

Data contrast analysis

This paper obtained camera pose data in CV-version and Sensor-version through the experiment, and compared the change trends of data increment, in order to analyze which version performs better in tracking registration.

User experience analysis

As shown in the Fig. 9, (1) the user holds the mobile horizontally to make the mobile camera aim at the front ground and performs the touch interaction; (2) the user wears the Google Glass to look at the front ground; (3) the user slides the Glass with fingers/clicks at height for functional interaction [50], [51]; (4) the user holds the cardboard to observe the front ground and can trigger the system function by pulling the magnet.

Gerhard Schall’s work inspired our research [11]. The 7-point

Conclusion

Based on ARGIS framework, this paper utilizes the Qingdao World Horticultural Expo underground pipeline data to design and implement the mobile augmented reality based underground pipeline information system including two versions: CV-version and Sensor-version. CV-version adopts the SFM-based registration and tracking algorithms, and uses the method TCNN to optimize the process of 3D feature matching, while Sensor-version adopts the multi-sensor processing based registration and tracking;

Acknowledgements

This research is supported by Innovation Fund for Technology Based Firms, China (No: 14c26211100180) and Qingdao Science and Technology project of China (No: 14-9-2-12-pt).

Reference (66)

  • G.R. King et al.

    ARVino — outdoor augmented reality visualisation of viticulture GIS data

  • M. Sun

    Hybrid tracking for augmented reality GIS registration

  • Y. Guo

    Application of augmented reality GIS in architecture

  • W.R. Gethin, A. Evans, H.D. Alan, The Use of Augmented Reality, GPS and INS for Subsurface Data,...
  • G. Schall

    Handheld augmented reality for underground infrastructure visualization

    Pers. Ubiquit. Comput.

    (2009)
  • G. Schall et al.

    North-centred orientation tracking on mobile phones

  • G. Schall et al.

    Smart vidente: advances in mobile augmented reality for interactive visualization of underground infrastructure

    Pers. Ubiquit. Comput.

    (2013)
  • X. Chang et al., Dynamic Concept Composition for Zero-Example Event Detection,...
  • A.H. Behzadan et al.

    Interactive augmented reality visualization for improved damage prevention and maintenance of underground infrastructure

  • A.J. Davison

    MonoSLAM: real-time single camera SLAM

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2007)
  • M.L. Pupilli et al.

    Real-time camera tracking using a particle filter

  • D. Chekhlov

    Robust real-time visual SLAM using scale prediction and exemplar based feature description

  • G. Klein et al.

    Parallel tracking and mapping for small AR workspaces

  • G. Klein et al.

    Parallel tracking and mapping on a camera phone

  • D. Wagner

    Real-time detection and tracking for augmented reality on mobile phones

    IEEE Trans. Visual. Comput. Graph.

    (2010)
  • D. Wagner et al.

    Multiple target detection and tracking with guaranteed framerates on mobile phones

  • A.S. Mavor et al.

    Virtual Reality: Scientific and Technological Challenges

    (1994)
  • S. You et al.

    Orientation tracking for outdoor augmented reality registration

    IEEE Comput. Graph. Appl.

    (1999)
  • P. Anke et al., Building a hybrid tracking system: integration of optical and magnetic tracking, in: iwar,...
  • Y. Genc

    Marker-less tracking for AR: a learning-based approach

  • ...
  • F.Z. Eishita et al.

    THEEMPA: simple AR games using layar

  • ...
  • Cited by (36)

    • BIM, machine learning and computer vision techniques in underground construction: Current status and future perspectives

      2021, Tunnelling and Underground Space Technology
      Citation Excerpt :

      Included are route design and site selection of utility networks and rock cavern (Cheng and Chang, 2001), digital recording of geological and geomechanical surveying data during tunnel excavation (Thum and Paoli, 2015), as well as supporting risk assessment of geohazards such as water inrush during tunnel construction (Li and Li, 2014). Moreover, GIS-based monitoring of tunnel deformation (Liu et al., 2009), underground mine subsidence (Li and Li, 2014), mining-induced surface deformation (Spreckels et al., 2001), underground pipeline surveying (Zhang et al., 2016b) and underground utilities mapping integrating augmented reality (Fenais et al., 2019) have also been implemented. In addition, Borrmann et al. (2015) proposed an IFC-based multi-scale tunnel model inspired by the CityGML’s intrinsic LoD representations.

    • A 3D dynamic visualization method coupled with an urban drainage model

      2019, Journal of Hydrology
      Citation Excerpt :

      Owing to its advantages in multidimensional space, it provides an advanced tool and method for coupling and parsing multi-source heterogeneous data (Liu et al., 2018; Trubka et al., 2016). In the field of urban drainage research, 3D visualization technology is at present predominantly used to show the distribution and attribute information of pipe networks (He et al., 2011; Zhang et al., 2016), and has yet to be combined with the urban drainage model. The information provided for analysis and decision making remains limited, and the capacity for data coupling and parsing are constrained (Breunig and Zlatanova, 2011).

    • Government affairs service platform for smart city

      2018, Future Generation Computer Systems
      Citation Excerpt :

      The 3D GIS provides a visual presentation and is used as a real 3D tool — for both queries by using a 3D model and visualizing geoinformation in 3D [16]. Our previous work based on WebVRGIS engine [17], which is WebGIS, 3D GIS and PPGIS, have proved the usability of 3D GIS for information management of e-government [18], including 3D interactive system for transportation [19], underground [20], water resources [21], virtual community [22]. By integrating the friendly interactive interface of Virtual Reality System and spatial analysis specialty of GIS, WebVRGIS is preferred in practical applications, especially in the geographical and urban planning.

    View all citing articles on Scopus

    This paper has been recommended for acceptance by Zicheng Liu.

    View full text