Hanif Fakhrurroja
ABSTRACT
This research discusses how to interact with a smart home using speech recognition and a touchscreen to control electronic devices. Google Speech Cloud API use to process speech-to-text and text-to-speech. The system is built in a mobile-based application using a touchscreen as remote control and speech to control the electronic devices. This mobile application is made using the Flutter framework. We use natural language understanding (NLU) in speech processing to determine the intent. The learning process in a dialogue system is based on reinforcement learning. Interaction through the touch screen on the mobile application performs well, while the dialogue system based on reinforcement learning accuracy rate is 83.33%.
- H. Fakhrurroja, A. Abdillah, U. Nadiya, and M. Arifin, ‘Hand state combination as gesture recognition using kinect v2 sensor for smart home control systems’, Proceedings - 2019 IEEE International Conference on Internet of Things and Intelligence System, IoTaIS 2019, pp. 74–78, 2019, doi: 10.1109/IoTaIS47347.2019.8980390.Google Scholar
Cross Ref
- M. Issam Younis and T. Fadhil Hussein, ‘Design and implementation of a contactless smart house network system’, International Journal of Electrical and Computer Engineering, vol. 8, no. 6, pp. 4663–4672, 2018, doi: 10.11591/ijece.v8i6.pp.4663-4672.Google Scholar
- S. Tharishny, S. Selvan, and P. Nair, ‘Android based smart house control via wireless communication’, International Journanl of Scientific Engineering and Technology, (5), no. 5, pp. 323–325, 2016.Google Scholar
- V. G. Sanchez, C. F. Pfeiffer, and N. O. Skeie, ‘A review of smart house analysis methods for assisting older people living alone’, Journal of Sensor and Actuator Networks, vol. 6, no. 3, pp. 1–38, 2017, doi: 10.3390/jsan6030011.Google ScholarCross Ref
- H. Fakhrurroja, Riyanto, A. Purwarianti, A. S. Prihatmanto, and C. Machbub, ‘Integration of Indonesian Speech and Hand Gesture Recognition for Controlling Humanoid Robot’, 2018 15th International Conference on Control, Automation, Robotics and Vision, ICARCV 2018, pp. 1590–1595, 2018, doi: 10.1109/ICARCV.2018.8581071.Google ScholarCross Ref
- S. S. Mousavi, M. Schukat, and E. Howley, ‘Deep Reinforcement Learning: An Overview’, Lecture Notes in Networks and Systems, vol. 16, pp. 426–440, 2018, doi: 10.1007/978-3-319-56991-8_32.Google Scholar
- A. Turnip, M. F. Amri, H. Fakrurroja, A. I. Simbolon, M. A. Suhendra, and D. E. Kusumandari, ‘Deception detection of EEG-P300 component classified by SVM method’, in Proceedings of the 6th International Conference on Software and Computer Applications - ICSCA ’17, Bangkok, Thailand, 2017, pp. 299–303. doi: 10.1145/3056662.3056709.Google ScholarDigital Library
- P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger, ‘Deep reinforcement learning that matters’, 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pp. 3207–3214, 2018.Google ScholarCross Ref
- H. Fakhrurroja, C. Machbub, A. S. Prihatmanto, and A. Purwarianti,2022. ‘Multimodal Fusion Algorithm and Reinforcement Learning-Based Dialog System in Human-Machine Interaction’, ijeei, vol. 12, no. 4, pp. 1016–1046, Dec. 2020, doi: 10.15676/ijeei.2020.12.4.19.Google Scholar
- J. C. A, R. Nagarajan, K. Satheeshkumar, N. Ajithkumar, P. A. Gopinath, and S. Ranjithkumar, ‘Intelligent Smart Home Automation and Security System Using Arduino and Wi-fi’, International Journal of Engineering And Computer Science, no. March, 2017, doi: 10.18535/ijecs/v6i3.53.Google Scholar
- S. Pirbhulal , ‘A novel secure IoT-based smart home automation system using a wireless sensor network’, Sensors (Switzerland), vol. 17, no. 1, pp. 1–19, 2017, doi: 10.3390/s17010069.Google Scholar
- S. Tang, V. Kalavally, K. Y. Ng, and J. Parkkinen, ‘Development of a prototype smart home intelligent lighting control architecture using sensors onboard a mobile computing system’, Energy and Buildings, vol. 138, pp. 368–376, 2017, doi: 10.1016/j.enbuild.2016.12.069.Google ScholarCross Ref
- H. Fakhrurroja, C. Machbub, A. S. Prihatmanto, and A. Purwarianti, ‘Multimodal Interaction System for Home Appliances Control’. International Journal of Interactive Mobile Technologies 14, no. 15, https://doi.org/10.3991/ijim.v14i15.13563.Google Scholar
- W. Wu, ‘React Native vs Flutter, cross-platform mobile application frameworks’, Metropolia University, no. March, p. 28, 2018.Google Scholar
- L. Dong , ‘Unified Language Model Pre-training for Natural Language Understanding and Generation’, no. NeurIPS, 2019.Google Scholar
- L. M. D. Da Silva, M. F. Torquato, and M. A. C. Fernandes, ‘Parallel Implementation of Reinforcement Learning Q-Learning Technique for FPGA’, IEEE Access, vol. 7, pp. 2782–2798, 2019, doi: 10.1109/ACCESS.2018.2885950.Google ScholarCross Ref
- J. Schulman, X. Chen, and P. Abbeel, ‘Equivalence Between Policy Gradients and Soft Q-Learning’, pp. 1–15, 2017.Google Scholar
- Yi and K. Jung, ‘A Chatbot by Combining Finite State Machine, Information Retrieval, and Bot-Initiative Strategy’, Alexa Price Proceedings, pp. 1–10, 2017.Google Scholar
- L. Moroney, The Definitive Guide to Firebase. 2017. doi: 10.1007/978-1-4842-2943-9.Google ScholarCross Ref
- W. J. Li, C. Yen, Y. S. Lin, S. C. Tung, and S. M. Huang, ‘JustIoT Internet of Things based on the Firebase real-time database’, Proceedings - 2018 IEEE International Conference on Smart Manufacturing, Industrial and Logistics Engineering, SMILE 2018, vol. 2018-Janua, pp. 43–47, 2018, doi: 10.1109/SMILE.2018.8353979.Google ScholarCross Ref
- Y. Amri and M. A. Setiawan, ‘Improving Smart Home Concept with the Internet of Things Concept Using RaspberryPi and NodeMCU’, IOP Conference Series: Materials Science and Engineering, vol. 325, no. 1, 2018, doi: 10.1088/1757-899X/325/1/012021.Google ScholarCross Ref
Recommendations
Sample-efficient batch reinforcement learning for dialogue management optimization
Spoken Dialogue Systems (SDS) are systems which have the ability to interact with human beings using natural language as the medium of interaction. A dialogue policy plays a crucial role in determining the functioning of the dialogue management module. ...
Reinforcement-learning based dialogue system for human-robot interactions with socially-inspired rewards
HighlightsWe integrate user appraisals in a POMDP-based dialogue manager procedure.We employ additional socially-inspired rewards in a RL setup to guide the learning.A unified framework for speeding up the policy optimisation and user adaptation.We ...
Learning cooperative persuasive dialogue policies using framing
Systems Performance for the user simulator is greatly improved by reinforcement learning.Framing is somewhat effective for the user simulator.Average rewards of system reach the minimum value with the policy where the estimated GPF reaches the highest ...
Comments