Abdelrahman Ahmad
ABSTRACT
In this paper, we introduce a detection system for macro worksteps in a manufacturing assembly line using depth images. The sensor is mounted on the ceiling with a top-down angle. The system was deployed in a real life industrial process where workers had to assemble an ATM machine. Experimental results show the effectiveness of three identification approaches that were used: (1) template matching using a single template per macro workstep, (2) multiple templates for macro worksteps and (3) template matching and motion detection in order to detect the transition between each two consecutive macro worksteps. Each approach has its own benefits in terms of processing speed, accuracy and precision and we discuss them in details along with the challenges the system had, in the discussion section. The results are also investigated in details and we present the future plans for the proposed detection system.
Supplemental Material
- Claus Bahlmann, Ying Zhu, Visvanathan Ramesh, Martin Pellkofer, and Thorsten Koehler. 2005. A system for traffic sign detection, tracking, and recognition using color, shape, and motion information. In IEEE Proceedings. Intelligent Vehicles Symposium, 2005. IEEE, 255--260.Google Scholar
Cross Ref
- Roberto Brunelli. 2009. Template matching techniques in computer vision: theory and practice. John Wiley & Sons.Google ScholarDigital Library
- Vincenzo Carletti, Luca Del Pizzo, Gennaro Percannella, and Mario Vento. 2017. An efficient and effective method for people detection from top-view depth cameras. In 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE, 1--6.Google ScholarCross Ref
- Raymond H Chan, Chung-Wa Ho, and Mila Nikolova. 2005. Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Transactions on image processing 14, 10 (2005), 1479--1485.Google ScholarDigital Library
- Tao Cheng, Jochen Teizer, Giovanni C Migliaccio, and Umberto C Gatti. 2013. Automated task-level activity analysis through fusion of real time location sensors and worker's thoracic posture data. Automation in Construction 29 (2013), 24--39.Google ScholarCross Ref
- Matthias Dantone, Juergen Gall, Christian Leistner, and Luc Van Gool. 2013. Human pose estimation using body parts dependent joint regressors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3041--3048.Google ScholarDigital Library
- Haibin Duan, Chunfang Xu, Senqi Liu, and Shan Shao. 2010. Template matching using chaotic imperialist competitive algorithm. Pattern recognition letters 31, 13 (2010), 1868--1875.Google Scholar
- Marcin Eichner, Manuel Marin-Jimenez, Andrew Zisserman, and Vittorio Ferrari. 2012. 2d articulated human pose estimation and retrieval in (almost) unconstrained still images. International journal of computer vision 99, 2 (2012), 190--214.Google ScholarDigital Library
- Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan. 2009. Object detection with discriminatively trained part-based models. IEEE transactions on pattern analysis and machine intelligence 32, 9 (2009), 1627--1645.Google Scholar
- Pedro F Felzenszwalb and Daniel P Huttenlocher. 2005. Pictorial structures for object recognition. International journal of computer vision 61, 1 (2005), 55--79.Google ScholarDigital Library
- Markus Funk, Lars Lischke, Sven Mayer, Alireza Sahami Shirazi, and Albrecht Schmidt. 2018. Teach Me How! Interactive Assembly Instructions Using Demonstration and In-Situ Projection. In Assistive Augmentation. Springer, 49--73.Google Scholar
- Klaus Greff, André Brandão, Stephan Krauß, Didier Stricker, and Esteban Clua. 2012. A comparison between background subtraction algorithms using a consumer depth camera.. In VISAPP (1). 431--436.Google Scholar
- Jungong Han, Eric J Pauwels, Paul M de Zeeuw, and Peter HN de With. 2012. Employing a RGB-D sensor for real-time tracking of humans across multiple re-entries in a smart environment. IEEE Transactions on Consumer Electronics 58, 2 (2012), 255--263.Google ScholarCross Ref
- Sven Hinrichsen, Daniel Riediger, and Alexander Unrau. 2016. Assistance systems in manual assembly. In Proceedings of 6th International conference on Production Engineering and Management, 29 September 2016. 3--14.Google Scholar
- Philipp Hold, Selim Erol, Gehard Reisinger, and Wilfried Sihn. 2017. Planning and evaluation of digital assistance systems. Procedia Manufacturing 9 (2017), 143--150.Google ScholarCross Ref
- Ninghang Hu, Gwenn Englebienne, and Ben Kröse. 2013. Posture recognition with a top-view camera. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2152--2157.Google Scholar
- Intel. [n.d.]. Stereo Depth - Intel® RealSense™ Depth and Tracking Cameras. Accessed: 2-10-2019.Google Scholar
- Ardalan Khosrowpour, Juan Carlos Niebles, and Mani Golparvar-Fard. 2014. Vision-based workface assessment using depth images for activity analysis of interior construction operations. Automation in Construction 48 (2014), 74--87.Google ScholarCross Ref
- Shu-Chun Lin, An-Sheng Liu, Tang-Wei Hsu, and Li-Chen Fu. 2015. Representative body points on top-view depth sequences for daily activity recognition. In 2015 IEEE international conference on systems, man, and cybernetics. IEEE, 2968--2973.Google ScholarDigital Library
- An-Sheng Liu, Zi-Jun Li, Tso-Hsin Yeh, Yu-Huan Yang, and Li-Chen Fu. 2017. Partially transferred convolution neural network with cross-layer inheriting for posture recognition from top-view depth camera. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 4139--4143.Google ScholarCross Ref
- Nan Lu, Jihong Wang, QH Wu, and Li Yang. 2008. An Improved Motion Detection Method for Real-Time Surveillance. IAENG International Journal of Computer Science 35, 1 (2008).Google Scholar
- Chinh Huu Pham, Quoc Khanh Le, and Thanh Ha Le. 2014. Human action recognition using dynamic time warping and voting algorithm. VNU Journal of Science: Computer Science and Communication Engineering 30, 3 (2014).Google Scholar
- Tuan Q Pham. 2010. Non-maximum suppression using fewer than two comparisons per pixel. In International Conference on Advanced Concepts for Intelligent Vision Systems. Springer, 438--451.Google ScholarCross Ref
- Michael Rauter. 2013. Reliable human detection and tracking in top-view depth images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 529--534.Google ScholarDigital Library
- Ehsan Rezazadeh Azar, Sven Dickinson, and Brenda McCabe. 2013. Server-customer interaction tracker: computer vision-based system to estimate dirt-loading cycles. Journal of Construction Engineering and Management 139, 7 (2013), 785--794.Google ScholarCross Ref
- Samy Sadeky, Ayoub Al-Hamadiy, Bernd Michaelisy, and Usama Sayed. 2010. Realtime automatic trafic accident recognition using hfg. In 2010 20th International Conference on Pattern Recognition. IEEE, 3348--3351.Google Scholar
- Chi-Chia Sun, Yi-Hua Wang, and Ming-Hwa Sheu. 2017. Fast motion object detection algorithm using complementary depth image on an RGB-D camera. IEEE Sensors Journal 17, 17 (2017), 5728--5734.Google ScholarCross Ref
- Ying-Li Tian and Arun Hampapur. 2005. Robust salient motion detection with complex background for real-time video surveillance. In 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05)-Volume 1, Vol. 2. IEEE, 30--35.Google ScholarDigital Library
- Silicon UK. [n.d.]. Tales In Tech History: Microsoft Kinect. Accessed: 05-01-2018.Google Scholar
- Yi-Hua Wang, Ming-Hwa Sheu, and Chi-Chia Sun. 2015. Eficient object motion detection based on RGB-D image. In 2015 IEEE International Conference on Consumer Electronics-Taiwan. IEEE, 438--439.Google ScholarCross Ref
- Wenming Yang, Wang Lu, and Naitong Zhang. 2007. Object extraction combining image partition with motion detection. In 2007 IEEE International Conference on Image Processing, Vol. 3. IEEE, III--337.Google ScholarCross Ref
- Hao Zhang, Christopher Reardon, and Lynne E Parker. 2013. Real-time multiple human perception with color-depth cameras on a mobile robot. IEEE Transactions on Cybernetics 43, 5 (2013), 1429--1441.Google ScholarCross Ref
Index Terms
- Macro workstep detection for assembly manufacturing
Recommendations
Privacy Preserving Workflow Detection for Manufacturing Using Neural Networks based Object Detection
IoT '21: Proceedings of the 11th International Conference on the Internet of ThingsIn this paper, we introduce a detection system for workflow in a manufacturing line using depth images to preserve the privacy of workers. A depth camera sensor is mounted on a ceiling with a top-down angle and pointed to workers below completing a ...
The Real-Time Eye Detection for Single User Based on Template Matching
CSE '14: Proceedings of the 2014 IEEE 17th International Conference on Computational Science and EngineeringA novel eye detection method based on template matching is proposed for glasses-free 3D device. Before matching, get the average eye template through a great quantity of eye images, splice several average templates into a chessboard template. Then ...
A Dual-Bound Algorithm for Very Fast and Exact Template Matching
Recently proposed fast template matching techniques employ rejection schemes derived from lower bounds on the match measure. This paper generalizes that idea and shows that in addition to lower bounds, upper bounds on the match measure can be used to ...
Comments