skip to main content
10.1145/3299815.3314440acmconferencesArticle/Chapter ViewAbstractPublication Pagesacm-seConference Proceedingsconference-collections
research-article

Voice and Motion-based Control System: Proof-of-Concept Implementation on Robotics via Internet-of-Things Technologies

Published: 18 April 2019 Publication History

Abstract

The Internet of Things (IoT) permits the connection of several devices to a network enabling data to be communicated over a network without direct human-to-human or human-to-computer communication. The core of IoT systems employs hardware devices, networks, cloud services and software applications. Hardware devices take input from outside the system, networks serve as a communication medium between hardware and the cloud server, cloud services store input data and software applications produce output using the cloud data. Because of the flexibility of IoT systems, it has become a popular model for modern device communication. IoT has a wide range of applications in a plethora of fields such as healthcare, manufacturing, home automation and education. To enhance usage of IoT for educators, this project is designed to implement an Arduino board alongside motion sensors and audio receiver to control a robot car by means of a cloud server, and IoT technologies. More specifically, an Arduino board will be trained using machine learning techniques to recognize a set of predefined gestures. These gestures will be stored in the cloud server, and the robot car will retrieve these commands from the cloud server reacting according to the gesture it the system has recognized. Moreover, the system integrates Google Voice API to control the robot car by pre-set voice commands. Furthermore, a web application is constructed to control the robot car by computer. This project is but a single yet powerful tool for future applications in smart classroom technologies.

References

[1]
C. Perera, C. H. Liu, S. Jayawardena and M. Chen, "A Survey on Internet of Things from Industrial Market Perspective," in IEEE Access, vol. 2, pp. 1660--1679, 2014.
[2]
J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, "Internet of Things (IoT): A Vision, Architectural Elements, and Future Directions", Future Generation Computing System, Vol. 29, pp. 1645--1660, 2013.
[3]
J. S. He, A. Atabekov, and H. M. Haddad, "Internet-of-Things Based Smart Resource Management System: A Case Study Intelligent Chair System", IEEE ICCCN 2016, Waikoloa, Hawaii, August 1-4, 2016.
[4]
J. S. He, A. Atabekov, and Y. Pan, "Implementing Internet of Things into Undergraduate Classes: Case Study a Capstone Research Project", Computer Education, 4(4), pp. 30--38, April 2016.
[5]
A. Atabekov, M. Starosielsky, D. Lo, and J. S. He, "Internet of Things-based Temperature Tracking System", IEEE COMPSAC 2015, Taichung, Taiwan, July 1-5, 2015.
[6]
G. Kiryakova, L. Yordanova, and N. Angelova, "Can We Make Schools and Universities Smarter with the Internet of Things?." TEM Journal, 6(1), pp. 80--84, 2017.
[7]
D. Jerkovi, R. Lacher, D. Stinson, and J. S. He, "An Energy-efficient Secure Adaptive Cloud-based Framework Supporting IoT Sensing Environment to Facilitate Big-Data Analysis and Visualization", 30th Anniversary National Conference on Undergraduate Research (NCUR), Asheville, NC, April 7-9, 2016.
[8]
J. S. He, D. Lo, Y. Xie, and J. Lartigue, "Integrating Internet of Things (IoT) into STEM Undergraduate Education: Case Study of a Modern Technology Infused Courseware for Embedded System Course", IEEE Frontiers in Education (FIE'16), Erie, PA, October 12-15, 2016.
[9]
J. S. He, S. Ji, and P. O. Bobbie, "Internet of Things (IoT)-based Learning Framework to Facilitate STEM Undergraduate Education", The Annual ACM Southeast Conference (ACMSE'17), Kennesaw, GA, April 13-15, 2017.
[10]
J. S. He, M. Han, and P. O. Bobbie, "An Energy-efficient Secure Adaptive Cloud-of-Things (CoT) Framework to Facilitate Undergraduate STEM Education", ACM SIGITE'18, Fort Lauderdale, Florida, October 3-6, 2018.
[11]
A. Atabekov, J. S. He, P. O. Bobbie, "Internet-of-Things-based Framework to Facilitate Indoor Localization Education", IEEE COMPSAC 2016, Atlanta, GA, June 10-14, 2016.
[12]
H. Liu, L. Wang, "Gesture Recognition for Human-robot Collaboration: A Review", International Journal of Industrial Ergonomics, Vol. 68, pp. 355--367, 2017.
[13]
N. Patel and J. S. He, "A Survey on Hand Gesture Recognition Techniques, Methods and Tools", International Journal of Research in Advent Technology (IJRAT), 6(6), June 2018.
[14]
D. Jackson, Z. Cesar, J. Matinez, M. Han, and J. S. He, "Motion-based Control System: Proof-of-Concept Implementation on Robotics via Internet-of-Things (IoT) Technologies", 2018 American Society for Engineering Education Southeastern Section Conference (ASEE-SE'18), Daytona Beach, FL, March 4-6, 2018.
[15]
M. Flores, C. Siloy, C. Oppus, and L. Agustin, "User-oriented Finger-gesture Glove Controller with Hand Movement Virtualization using Flex Sensors and a Digital Accelerometer". 2014 International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM), Palawan, Philippines, November 12-16, 2014.
[16]
A. Bhuyan and T. Mallick, "Gyro-accelerometer based Control of a Robotic Arm using AVR Microcontroller", 9th International Forum on Strategic Technology (IFOST), Strategic Technology (IFOST), Cox's Bazar, Bangladesh, October 21-23, 2014.
[17]
Arduino, https://www.arduino.cc.
[18]
Arduino Products, https://www.arduino.cc/en/Main/Products.
[19]
Raspberry Pi Products, https://www.raspberrypi.org.
[20]
PLXDAQ. https://www.parallax.com/downloads/plx-daq.
[21]
Raspberry Sense HAT, https://projects.raspberrypi.org/en/projects/getting-stated-with-the-sense-hat.
[22]
AWS IoT, https://docs.aws.amazon.com/aws-technicial-content/latest/aws-overview/internet-of-things-services.html.
[23]
P. Zhong and M. Fukushima, "A New Multi-Class Support Vector Algorithm, Optimization Methods and Software", 21(3), pp. 359--372, 2006.

Cited By

View all
  • (2024)Voice-Controlled Human-Machine Interface for an Assistive Exoskeleton Glove Aiding Patients with Brachial Plexus Injuries2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)10.1109/RO-MAN60168.2024.10731436(1907-1913)Online publication date: 26-Aug-2024
  • (2023)Design and Implement Services Robot Based on Intelligent Controller with IoT Techniques2023 16th International Conference on Developments in eSystems Engineering (DeSE)10.1109/DeSE60595.2023.10469212(13-18)Online publication date: 18-Dec-2023
  • (2023)A Novel and Lightweight Real-Time Continuous Motion Gesture Recognition Algorithm for SmartphonesIEEE Access10.1109/ACCESS.2023.325540211(42725-42737)Online publication date: 2023
  • Show More Cited By

Index Terms

  1. Voice and Motion-based Control System: Proof-of-Concept Implementation on Robotics via Internet-of-Things Technologies

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          ACMSE '19: Proceedings of the 2019 ACM Southeast Conference
          April 2019
          295 pages
          ISBN:9781450362511
          DOI:10.1145/3299815
          • Conference Chair:
          • Dan Lo,
          • Program Chair:
          • Donghyun Kim,
          • Publications Chair:
          • Eric Gamess
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 18 April 2019

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. Arduino
          2. Cloud Server
          3. Gesture Recognition
          4. Google Voice API
          5. Internet of Things
          6. Machine Learning
          7. Motion-based Control
          8. Raspberry Pi
          9. Robotics
          10. Voice-based Control

          Qualifiers

          • Research-article
          • Research
          • Refereed limited

          Conference

          ACM SE '19
          Sponsor:
          ACM SE '19: 2019 ACM Southeast Conference
          April 18 - 20, 2019
          GA, Kennesaw, USA

          Acceptance Rates

          Overall Acceptance Rate 502 of 1,023 submissions, 49%

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)16
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 15 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2024)Voice-Controlled Human-Machine Interface for an Assistive Exoskeleton Glove Aiding Patients with Brachial Plexus Injuries2024 33rd IEEE International Conference on Robot and Human Interactive Communication (ROMAN)10.1109/RO-MAN60168.2024.10731436(1907-1913)Online publication date: 26-Aug-2024
          • (2023)Design and Implement Services Robot Based on Intelligent Controller with IoT Techniques2023 16th International Conference on Developments in eSystems Engineering (DeSE)10.1109/DeSE60595.2023.10469212(13-18)Online publication date: 18-Dec-2023
          • (2023)A Novel and Lightweight Real-Time Continuous Motion Gesture Recognition Algorithm for SmartphonesIEEE Access10.1109/ACCESS.2023.325540211(42725-42737)Online publication date: 2023
          • (2022)Robust Real-time Automatic Voice Command based on Raspberry pi for assistance disabled people2022 IEEE 9th International Conference on Sciences of Electronics, Technologies of Information and Telecommunications (SETIT)10.1109/SETIT54465.2022.9875545(474-478)Online publication date: 28-May-2022
          • (2020)Voice Communication in Noisy Environments in a Smart House Using Hybrid LMS+ICA AlgorithmSensors10.3390/s2021602220:21(6022)Online publication date: 23-Oct-2020

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media