skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: ODDS: Real-Time Object Detection Using Depth Sensors on Embedded GPUs

Conference · · 2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN)
; ; ;

Detecting objects that are carried when someone enters or exits a room is very useful for a wide range of smart building applications including safety, security, and energy efficiency. While there has been a significant amount of work on object recognition using large-scale RGB image datasets, RGB cameras are too privacy invasive in many smart building applications and they work poorly in the dark. Additionally, deep object detection networks require powerful and expensive GPUs. We propose a novel system that we call ODDS (Object Detector using a Depth Sensor) that can detect objects in real-time using only raw depth data on an embedded GPU, e.g., NVIDIA Jetson TX1. Hence, our solution is significantly less privacy invasive (even if the sensor is compromised) and less expensive, while maintaining a comparable accuracy with state of the art solutions. Specifically, we resort to training a deep convolutional neural network using raw depth images, with curriculum based learning to improve accuracy by considering the complexity and imbalance in object classes and developing a sparse coding based technique that speeds up the system ~2× with minimal loss of accuracy. Based on a complete implementation and real-world evaluation, we see ODDS achieve 80.14% mean average precision in object detection in real-time (5-6 FPS) on a Jetson TX1.

Research Organization:
Robert Bosch LLC, Farmington Hills, MI (United States)
Sponsoring Organization:
USDOE Office of Energy Efficiency and Renewable Energy (EERE), Energy Efficiency Office. Building Technologies Office
DOE Contract Number:
EE0007682
OSTI ID:
1811681
Journal Information:
2018 17th ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN), Conference: ACM/IEEE Conference on Information Processing in Sensor Networks (IPSN) , Porto, Portugal, April 11-13, 2018.
Country of Publication:
United States
Language:
English

References (29)

Indoor Person Identification through Footstep Induced Structural Vibration conference January 2015
FORK: fine grained occupancy estimatoR using kinect on ARM embedded platforms
  • Munir, Sirajum; Tran, Le; Francis, Jonathan
  • BuildSys '17: The 4th ACM International Conference on Systems for Energy-Efficient Built Environments, Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments https://doi.org/10.1145/3137133.3141461
conference November 2017
Fast ConvNets Using Group-Wise Brain Damage conference June 2016
Real-Time Fine Grained Occupancy Estimation Using Depth Sensors on ARM Embedded Platforms conference April 2017
Generating Diverse Image Datasets with Limited Labeling conference October 2016
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression conference October 2017
Sonicdoor
  • Khalil, Nacer; Benhaddou, Driss; Gnawali, Omprakash
  • Proceedings of the 4th ACM International Conference on Systems for Energy-Efficient Built Environments https://doi.org/10.1145/3137133.3137154
conference November 2017
3D through-wall imaging with unmanned aerial vehicles using wifi conference April 2017
Cross Modal Distillation for Supervision Transfer conference June 2016
Channel Pruning for Accelerating Very Deep Neural Networks conference October 2017
Caffe: Convolutional Architecture for Fast Feature Embedding conference January 2014
Forma Track
  • Kalyanaraman, Avinash; Hong, Dezhi; Soltanaghaei, Elahe
  • Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 1, Issue 3 https://doi.org/10.1145/3130926
journal September 2017
Going deeper with convolutions conference June 2015
Discriminative Multi-modal Feature Fusion for RGBD Indoor Scene Recognition conference June 2016
DeepIoT conference November 2017
Scpl conference April 2013
Unsupervised Learning of Visual Representations Using Videos conference December 2015
Curriculum learning conference January 2009
Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers book January 2011
Towards Scalable Summarization of Consumer Videos Via Sparse Dictionary Selection journal February 2012
Histograms of Oriented Gradients for Human Detection conference January 2005
See all by looking at a few: Sparse modeling for finding representative objects conference June 2012
Indoor Scene Understanding with RGB-D Images: Bottom-up Segmentation, Object Detection and Semantic Segmentation journal November 2014
Visual tracking with online Multiple Instance Learning conference June 2009
Low-Power Radio-Optical Beacons for In-View Recognition conference September 2015
Second-order constrained parametric proposals and sequential search-based structured prediction for semantic segmentation in RGB-D images conference June 2015
SUN RGB-D: A RGB-D scene understanding benchmark suite conference June 2015
You Only Look Once: Unified, Real-Time Object Detection conference June 2016
Curriculum learning of multiple tasks conference June 2015

Similar Records

FORK: fine grained occupancy estimatoR using kinect on ARM embedded platforms
Conference · Wed Nov 08 00:00:00 EST 2017 · OSTI ID:1811681
+7 more
Real-Time Fine Grained Occupancy Estimation Using Depth Sensors on ARM Embedded Platforms
Conference · Tue Apr 18 00:00:00 EDT 2017 · 2017 IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS) · OSTI ID:1811681
+6 more
Integrate Light-Weight Deep Learning Tools with Internet of Things
Technical Report · Mon Sep 30 00:00:00 EDT 2019 · OSTI ID:1811681

Related Subjects

29 ENERGY PLANNING, POLICY, AND ECONOMY
Object detection
human detection
depth sensor