Abstract
Smart homes can provide complementary information to assist home service robots. We present a robotic misplaced item finding (MIF) system, which uses human historical trajectory data obtained in a smart home environment. First, a multi-sensor fusion method is developed to localize and track a resident. Second, a path-planning method is developed to generate the robot movement plan, which considers the knowledge of the human historical trajectory. Third, a real-time object detector based on a convolutional neural network is applied to detect the misplaced item. We present MIF experiments in a smart home testbed and the experimental results verify the accuracy and efficiency of our solution.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahern SJ, Carter J, Wilson P, 2015. Autonomous robot retrieval system. SAI Intelligent Systems Conf, p.280–282. https://doi.org/10.1109/IntelliSys.2015.7361155
Castellanos JA, Neira J, Tardós JD, 2001. Multisensor fusion for simultaneous localization and map building. IEEE Trans Rob Autom, 17(6):908–914. https://doi.org/10.1109/70.976024
Choi YS, Deyle T, Chen T, et al., 2009. A list of household objects for robotic retrieval prioritized by people with ALS. IEEE Int Conf on Rehabilitation Robotics, p.510–517. https://doi.org/10.1109/ICORR.2009.5209484
Ekvall S, Kragic D, Jensfelt P, 2007. Object detection and mapping for service robot tasks. Robotica, 25(2):175–187. https://doi.org/10.1017/S0263574706003237
Fujii T, Ueda H, Minoh M, 2007. A looking-for-objects service in ubiquitous home. J Nat Inst Inform Commun Technol, 54(3):175–181.
Huynh SM, Parry D, Fong ACM, et al., 2014. Home localization system for misplaced objects. IEEE Int Conf on Consumer Electronics, p.462-463. https://doi.org/10.1109/ICCE.2014.6776087
Kollar T, Roy N, 2009. Utilizing object-object and object-scene context when planning to find things. IEEE Int Conf on Robotics and Automation, p.2168–2173. https://doi.org/10.1109/ROBOT.2009.5152831
Komatsuzaki M, Tsukada K, Siio I, et al., 2011. IteMinder: finding items in a room using passive RFID tags and an autonomous robot (poster). Proc 13th Int Conf on Ubiquitous Computing, p.599–600. https://doi.org/10.1145/2030112.2030232
Lin YC, Wei ST, Yang SA, et al., 2015. Planning on searching occluded target object with a mobile robot manipulator. IEEE Int Conf on Robotics and Automation, p.3110–3115. https://doi.org/10.1109/ICRA.2015.7139626
Meng Q, Lee MH, 2006. Design issues for assistive robotics for the elderly. Adv Eng Inform, 20(2):171–186. https://doi.org/10.1016/j.aei.2005.10.003
PIR, 2018. Passive Infrared Sensor. https://en.wikipedia.org/wiki/Passive_infrared_sensor
Ravi K, Bhavani KJ, Vinayak M, et al., 2016. Identification of misplaced objects. Indian J Sci Technol, 9(17):1–4. https://doi.org/10.17485/ijst/2016/v9i17/92991
Redmon J, Divvala S, Girshick R, et al., 2016. You only look once: unified, real-time object detection. Proc IEEE Conf on Computer Vision and Pattern Recognition, p.779–788. https://doi.org/10.1109/CVPR.2016.91
Rogers JG, Christensen HI, 2013. Robot planning with a semantic map. IEEE Int Conf on Robotics and Automation, p.2239-2244. https://doi.org/10.1109/ICRA.2013.6630879
ROS, 2018. Robot Operating System. http://www.ros.org/
Sadhu C, Abhiram MH, Chandan B, et al., 2013. Cognitive learning enabled real time object search robot. Int Conf on Control, Automation, Robotics and Embedded System, p.1–6. https://doi.org/10.1109/CARE.2013.6733739
Schwarz D, Schwarz M, Stückler J, et al., 2014. Cosero, find my keys! Object localization and retrieval using bluetooth low energy tags. Proc 18th RoboCup Int Symp, p.195–206. https://doi.org/10.1007/978-3-319-18615-3_16
Susi M, Renaudin V, Lachapelle G, 2013. Motion mode recognition and step detection algorithms for mobile phone users. Sensors, 13(2):1539–1562. https://doi.org/10.3390/s130201539
Tesoriero R, Gallud JA, Lozano MD, et al., 2009. Tracking autonomous entities using RFID technology. IEEE Trans Consum Electron, 55(2):650–655. https://doi.org/10.1109/TCE.2009.5174435
Wang Q, 2018. Youtube Video on Robotic Misplaced Item Finding System. https://youtu.be/jEDKbr71pis
Wang Q, Zhang SL, Liu MQ, et al., 2017. Retrieval of misplaced items using a mobile robot via visual object recognition. IEEE 7th Conf on CYBER Technology in Automation, Control, and Intelligent Systems, p.1188–1193. https://doi.org/10.1109/CYBER.2017.8446158
Xu D, Huang Q, Liu HW, 2016. Object detection on robot operation system. IEEE 11th Conf on Industrial Electronics and Applications, p.1155–1159. https://doi.org/10.1109/ICIEA.2016.7603758
Xunfei, 2018. Voice Dictation. http://www.xfyun.cn/services/voicedictation
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Qi WANG, Zhen FAN, Wei-hua SHENG, Sen-lin ZHANG, and Mei-qin LIU declare that they have no conflict of interest.
Additional information
Project supported by the Basic Public Research Program of Zhejiang Province, China (No. LGF18F030001) and the Open Research Project of the State Key Laboratory of Industrial Control Technology, Zhejiang University, China (No. ICT1800414)
Rights and permissions
About this article
Cite this article
Wang, Q., Fan, Z., Sheng, Wh. et al. Finding misplaced items using a mobile robot in a smart home environment. Frontiers Inf Technol Electronic Eng 20, 1036–1048 (2019). https://doi.org/10.1631/FITEE.1800275
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.1800275