Abstract
Self-localization is a fundamental requirement for autonomous mobile robots. With the rapid development in sensor technology, the sensor suites of robot provide multimodal information that naturally ensures perception robustness, multimodal sensory fusion are able to provide a better solution for enhance the capability of self-localization. This paper proposes a multimodal sensory fusion method based on Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN) for RoboCup 3D Simulation league. This approach fuses Inertia Navigation System (INS) and vision perceptor information from different sensors at feature level instead of raw data. The experiment results demonstrate that the proposed approach makes an improvement in predictive accuracy and efficiency compared with the standard Extended Kalman Filter (EKF) and the static Particle Filter (PF) methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Aguirre E, Garcia-Silvente M, Pascual D (2016) A multisensor based approach using supervised learning and particle filtering for people detection and tracking. In: Robot 2015: Second Iberian Robotics Conference, pp 645–657. doi:10.1007/978-3-319-27149-1_50
Chen SY (2012) Kalman filter for robot vision: a survey. IEEE Trans Indus Electron 59(11):4409–4420. doi:10.1109/TIE.2011.2162714
Deng BZ, Miao H, Gen L et al (2011) Self-localization of omni-directional soccer robot based on multi-sensor. Control Theory Appl 28(12):1821–1824. http://jcta.alljournals.ac.cn/ct_cn/ch/reader/create_pdf.aspx?file_no=CCTA090911&year_id=2011&quarter_id=12&falg=1
Djuric PM, Kotecha JH, Zhang J et al (2003) Particle filtering. Signal Process Mag IEEE 20(5):19–38. doi:10.1109/MSP.2003.1236770
Fox D, Hightower J, Liao L et al (2003) Bayesian filtering for location estimation. Pervasive Comput IEEE 2(3):24–33. doi:10.1109/MPRV.2003.1228524
Gers FA, Eck D, Schmidhuber J (2001) Applying LSTM to time series predictable through time-window approaches. In: International Conference on Artificial Neural Networks, pp 669-676. doi:10.1007/3-540-44668-0_93
Gers FA, Schraudolph NN, Schmidhuber J et al (2003) Learning precise timing with LSTM recurrent networks. J Mach Learn Res 3(1):115–143. doi:10.1162/153244303768966139
Gouaillier D, Hugel V, Blazevic P et al (2009) Mechatronic design of NAO humanoid. In: IEEE International Conference on Robotics and Automation, pp 769–774. doi:10.1109/ROBOT.2009.5152516
Hashemi E, Jadid MG, Lashgarian M et al (2012) Particle filter based localization of the Nao biped robots. In: 44th Southeastern Symposium on System Theory (SSST), pp 168–173. doi:10.1109/SSST.2012.6195128
Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. doi:10.1162/neco.1997.9.8.1735
Kam M, Zhu X, Kalata P (1997) Sensor fusion for mobile robot navigation. Proc IEEE 85(1):108–119. doi:10.1109/JPROC.1997.554212
Kim J, Kim Y, Kim S (2008) An accurate localization for mobile robot using extended Kalman filter and sensor fusion. In: 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp 2928–2933. doi:10.1109/IJCNN.2008.4634210
Kogler M, Obst O (2003) Simulation league: the next generation. In: RoboCup 2003: Robot Soccer World Cup VII, pp 458–469. doi:10.1007/978-3-540-25940-4_40
Koval V, Adamiv O, Kapura V (2007) The local area map building for mobile robot navigation using ultrasound and infrared sensors. In: IEEE Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, pp 454–459. doi:10.1109/IDAACS.2007.4488459
Leonard JJ, Durrantwhyte HF (1991) Mobile robot localization by tracking geometric beacons. IEEE Trans Robot Autom 7(3):376–382. doi:10.1109/70.88147
Liang Z, Hao Y (2014) Adaptive particle filter for self-localization of RoboCup 3D soccer robots. Int J Robot Autom 29(3): doi:10.2316/Journal.206.2014.3.206-3969
Luo RC, Chang CC (2012) Multisensor fusion and integration: a review on approaches and its applications in mechatronics. IEEE Trans Ind Inf 8(1):49–60. doi:10.1109/TII.2011.2173942
Rodriguez FSA, Fremont V, Bonnifait P (2009) An experiment of a 3D real-time robust visual odometry for intelligent vehicles. In: International IEEE Conference on Intelligent Transportation Systems, pp 1–6. doi:10.1109/ITSC.2009.5309615
Smith D, Singh S (2006) Approaches to multisensor data fusion in target tracking: a survey. IEEE Trans Knowl Data Eng 18(12):1696–1710. doi:10.1109/TKDE.2006.183
Stoecker J, Visser U (2012) RoboViz: programmable visualization for simulated soccer. In: RoboCup 2011: Robot Soccer World Cup XV, pp 282–293. doi:10.1007/978-3-642-32060-6_24
Thrun S (2002) Probabilistic robotics. Commun Acm 45(3):52–57. doi:10.1145/504729.504754
Titterton D (2004) Strapdown inertial navigation technology. Aerosp Electron Syst Mag IEEE 20(7):33–34. doi:10.1049/PBRA017E
Acknowledgements
The research is supported by part of the National Natural Science Foundation (Surface Project No. 61304250).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lu, W., Zhang, J., Zhao, X. et al. Multimodal sensory fusion for soccer robot self-localization based on long short-term memory recurrent neural network. J Ambient Intell Human Comput 8, 885–893 (2017). https://doi.org/10.1007/s12652-017-0483-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12652-017-0483-7