Abstract
Many robot applications, such as environmental monitoring, security and surveillance help people to do tasks in day-to-day scenarios. However, the growing security demand for environment perception is a key issue of mapping or frequent updating in the long term, such as fire detection in early stage. A hybrid mapping method is proposed based on fusing RGB, depth and thermal (DT) information from Kinect and infrared sensors equipped in the mobile robot. Firstly, the proposed pipeline will estimate the robot’s pose by extracting and matching ORB features in RGB images successively. Then Poses corresponding to each depth and thermal- Infrared image are estimated through a combination of timestamp synchronization and the result of the extrinsic calibration of the system, and the map with both appearance and the temperature of environment is generated by the combination of The RGB and temperature information. Finally, the depth information is used to project the pixel points to the world coordinate system to generate the RGB-DT map. Extensive results verify the effectiveness of the proposed RGB-DT mapping for environments perception.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tan, Y.: A survey on visual perception for firefighting robots. J. Mianyang Teach. Coll. 2, 40–45 (2018)
Newcombe, R.A., Izadi, S., et al.: KinectFusion: real-time dense surface mapping and tracking. In: IEEE International Symposium on Mixed and Augmented Reality. pp. 127–136. IEEE, Basel (2011)
Dai, A., Izadi, S., Theobalt, C.: BundleFusion: real-time globally consistent 3D reconstruction using on-the-fly surface re-integration. ACM Trans. Graph. 36(4), 76a (2017)
Mur-Artal, R., Tardós, J.D.: ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)
Vidas, S., Moghadam, P., Bosse, M.: 3D thermal mapping of building interiors using an RGB-D and thermal camera. In: IEEE International Conference on Robotics and Automation, pp. 2311–2318. IEEE, Karlsruhe (2013)
Borrmann, D., Nüchter, A., Djakulovi’C, M., et al.: The project thermal mapper-thermal 3D mapping of indoor environments for saving energy. In: International IFAC Symposium on Robot Control, pp. 31–38 (2012)
Nagatani, K., Otake, K., Yoshida, K.: Three-dimensional thermography mapping for mobile rescue robots. In: Yoshida, K., Tadokoro, S. (eds.) Field and Service Robotics, pp. 49–63. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-40686-7_4
Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)
Zhang, Q., Pless, R.: Extrinsic calibration of a camera and laser range finder (improves camera calibration). In: IEEE/RSJ International Conference on Intelligent Robots & Systems. IEEE, Sendai (2005)
Moghadam, P., Bosse, M., Zlot, R.: Line-based extrinsic calibration of range and image sensors. In: 2013 IEEE International Conference on Robotics and Automation. IEEE, Karlsruhe (2013)
Hwang, S., Choi, Y., Kim, N., et al.: Low-cost synchronization for multispectral cameras. In: International Conference on Ubiquitous Robots & Ambient Intelligence. IEEE, Goyang (2015)
Baar, J.V., Beardsley, P., Pollefeys, M., et al.: Sensor fusion for depth estimation, including TOF and thermal sensors. In: 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, pp. 472–478. IEEE Computer Society, Zurich (2012)
Ge, P., Yang, B., Han, Q., et al.: Infrared image detail enhancement algorithm based on hierarchical processing by guided image filter. Infrared Technol. 40(12), 45–53 (2018)
Chang, H., Chen, C.: Image fusion based on HSV color space model and wavelet transform. Comput. Eng. Des. 28(23), 5682–5684 (2007)
Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate O (n) solution to the PnP problem. Int. J. Comput. Vis. 81(2), 155–166 (2009)
Galvez-Lo, P.D., Tardos, J.D.: Bags of binary words for fast place recognition in image sequences. IEEE Trans. Robot. 28(5), 1188–1197 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, L., Liu, Y., Jiang, X., Wang, K., Zhou, Z. (2019). Indoor Environment RGB-DT Mapping for Security Mobile Robots. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds) Intelligent Robotics and Applications. ICIRA 2019. Lecture Notes in Computer Science(), vol 11743. Springer, Cham. https://doi.org/10.1007/978-3-030-27538-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-27538-9_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27537-2
Online ISBN: 978-3-030-27538-9
eBook Packages: Computer ScienceComputer Science (R0)