Authors:
Jin Chen
1
and
Zhigang Zhu
2
;
1
Affiliations:
1
Visual Computing Laboratory/Data Science and Engineering Program, Computer Science Department, The City College of New York - CUNY, New York, NY 10031 U.S.A.
;
2
PhD Program in Computer Science, The Graduate Center - CUNY, New York, NY 10016, U.S.A.
Keyword(s):
3D Object Detection, 3D Object Tracking, Obstacle Detection, Assistive Computer Vision.
Abstract:
Real-time detection of 3D obstacles and recognition of humans and other objects is essential for blind or low- vision people to travel not only safely and independently but also confidently and interactively, especially in a cluttered indoor environment. Most existing 3D obstacle detection techniques that are widely applied in robotic applications and outdoor environments often require high-end devices to ensure real-time performance. There is a strong need to develop a low-cost and highly efficient technique for 3D obstacle detection and object recognition in indoor environments. This paper proposes an integrated 3D obstacle detection system implemented on a smartphone, by utilizing deep-learning-based pre-trained 2D object detectors and ARKit- based point cloud data acquisition to predict and track the 3D positions of multiple objects (obstacles, humans, and other objects), and then provide alerts to users in real time. The system consists of four modules: 3D obstacle detection, 3D
object tracking, 3D object matching, and information filtering. Preliminary tests in a small house setting indicated that this application could reliably detect large obstacles and their 3D positions and sizes in the real world and small obstacles’ positions, without any expensive devices besides an iPhone.
(More)