Abstract
With the advancements of automation technologies, factory operations have shifted from labor-based to semi-automatic and fully automatic and may even become unmanned in the future. Therefore, the conditions of a factory can be monitored from a distance and the machines can be remotely controlled. This study investigated the predictive distant operation of a computer numerical control (CNC) machine virtually controlled with hand gestures. Similar previous studies were conducted only involving robotic arms. Nevertheless, the attempt in this study is a crucial step toward entirely Internet- or cloud-based manufacturing. In the proposed methodology, meaningful hand gestures corresponding to eight commonly used CNC operations were established. Kinect was then used to track the skeletal information of an operator, and based on this information, the joint angles of the operator were derived using the space vector method. Four rules were then proposed to predict the hand gesture of the operator according to the changes of the joint angles when making the hand gestures to control the CNC machine. An experimental system was established to demonstrate the applicability of the proposed methodology. According to the experimental results, the eight hand gestures were recognized with favorable accuracy. In addition, analysis of variance was performed and the results indicated that the performance of the proposed methodology is robust to the operators’ differences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Archimede, B., Letouzey, A., Memon, M. A., & Xu, J. (2014). Towards a distributed multi-agent framework for shared resources scheduling. Journal of Intelligent Manufacturing, 25(5), 1077–1087.
Binney, D., & Boehm, J. (2010). Performance evaluation of the PrimeSense IR projected pattern depth sensor. http://www-research.cege.ucl.ac.uk/Posters/2011PosterFair/88_Binney_Daniel.pdf.
Borangiu, T., Raileanu, S., Trentesaux, D., Berger, T., & Iacob, I. (2014). Distributed manufacturing control with extended CNP interaction of intelligent products. Journal of Intelligent Manufacturing, 25(5), 1065–1075.
Chen, T. (2014). Strengthening the competitiveness and sustainability of a semiconductor manufacturer with cloud manufacturing. Sustainability, 6, 251–268.
Dang, Q.-V., Nielsen, I., Steger-Jensen, K., & Madsen, O. (2014). Scheduling a single mobile robot for part-feeding tasks of production lines. Journal of Intelligent Manufacturing, 25(6), 1271–1287.
Du, G., Zhang, P., Mai, J., & Li, Z. (2012). Markerless Kinect-based hand tracking for robot teleoperation. International Journal of Advanced Robotic Systems, 9, 36.
Fan, Y., Zhao, D., Zhang, L., Huang, S., & Liu, B. (2004). Manufacturing grid: Needs, concept and architecture. Lecture Notes in Computer Science, 3032, 653–656.
Gama, A. D., Chaves, T., Figueiredo, L., & Teichrieb, V. (2011). Improving motor rehabilitation process through a natural interaction based system using Kinect sensor. In IEEE symposium on 3D user interfaces (pp. 145–146).
Gil, P., Mateo, C., & Torres, F. (2014). 3D visual sensing of the human hand for the remote operation of a robotic hand. International Journal of Advanced Robotic Systems, 11, 26.
Hsieh, F.-S., & Lin, J.-B. (2014). Context-aware workflow management for virtual enterprises based on coordination of agents. Journal of Intelligent Manufacturing, 25(3), 393–412.
Huang, T. S., & Pavlovic, V. I. (1995). Hand gesture modeling, analysis and synthesis. In Proceedings of international workshop on automatic face and gesture recognition (pp. 73–79).
Jeon, M.-J., Lee, S. W., & Bien, Z. (2011). Hand gesture recognition using multivariate fuzzy decision tree and user adaptation. International Journal of Fuzzy System Applications, 1(3), 15–31.
Lambrecht, J., Kleinsorge, M., Rosenstrauch, M., & Krüger, J. (2013). Spatial programming for industrial robots through task demonstration. International Journal of Advanced Robotics Systems, 10, 254.
Lange, B., Koenig, S., McConnell, E., Chang, C. Y., Juang, R., Suma, E., Bolas, M. , & Rizzo, A. (2012). Interactive game-based rehabilitation using the Microsoft Kinect. In IEEE virtual reality (pp. 171–172).
Li, B. H., Zhang, L., & Wang, S. L. (2010). Cloud manufacturing: A service-oriented new networked manufacturing model. Computer Integrated Manufacturing Systems, 16(1), 1–9.
Lin, Z. (2014). Distant operation and virtual control of CNC machines. Master Thesis, Taiwan:Feng Chia University,
Microsoft TechNet. (2014). Working of Microsoft’s PrimeSense technology based Kinect: An elaboration. http://social.technet.microsoft.com/wiki/contents/articles/6370.working-of-microsofts-primesense-technology-based-kinect-an-elaboration.aspx.
Microsoft. (2014). Kinect for Windows. http://www.microsoft.com/en-us/kinectforwindows/.
OpenKinect.org. (2012). Welcome to the OpenKinect project. http://openkinect.org/wiki/Main_Page.
Pham, A. (2009). E3: Microsoft shows off gesture control technology for Xbox 360. Los Angeles Times.
Sarraj, R., Ballot, E., Pan, S., & Montreuil, B. (2014). Analogies between Internet network and logistics service networks: Challenges involved in the interconnection. Journal of Intelligent Manufacturing, 25(6), 1207–1219.
Smid, P. (2008). CNC programming handbook (3rd ed.). New York: Industrial Press.
Smid, P. (2010). CNC control setup for milling and turning. New York: Industrial Press.
Valentino, J., & Goldenberg, J. (2003). Introduction to computer numerical control (CNC). Englewood Cliffs: Prentice Hall.
Wu, D., Greer, M. J., Rosen, D. W., & Schaefer, D. (2013a). Cloud manufacturing: Strategic vision and state-of-the-art. Journal of Manufacturing Systems, 32(4), 564–579.
Wu, D., Thames, J. L., Rosen, D. W., & Schaefer, D. (2013b). Enhancing the product realization process with cloud-based design and manufacturing systems. Journal of Computing and Information Science in Engineering, 13(4), 1–14.
Wu, Q., Zhu, Q., & Zhou, M. (2014). A correlation-driven optimal service selection approach for virtual enterprise establishment. Journal of Intelligent Manufacturing, 25(6), 1441–1453.
Wu, D., Rosen, D. W., Wang, L., & Schaefer, D. (2015). Cloud-based design and manufacturing: A new paradigm in digital manufacturing and design innovation. Computer-Aided Design, 59, 1–14.
Yang, S., Park, K., & Bien, Z. (2011). Gesture spotting using fuzzy garbage model and user adaptation. International Journal of Fuzzy System Applications, 1(3), 47–65.
Zhang, L., Guo, H., Tao, F., Luo, Y. L., & Si, N. (2010). Flexible management of resource service composition in cloud manufacturing. IEEE International Conference on Industrial Engineering and Engineering Management (pp. 2278–2282).
Zhenyao, M., & Neumann, U. (2006). Real-time hand pose recognition using low-resolution depth images. Proceedings of the 2006 IEEE computer society conference on computer vision and pattern recognition (Vol. 2, pp. 1499–1505).
Acknowledgments
This work was supported by Ministry of Science and Technology of Taiwan under the Grant MOST 103-2221-E-035-073-MY3.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, T., Wang, YC. & Lin, Z. Predictive distant operation and virtual control of computer numerical control machines. J Intell Manuf 28, 1061–1077 (2017). https://doi.org/10.1007/s10845-014-1029-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-014-1029-x