Junaid Younas
Editorial Notes
The authors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected VoR was published on September 29, 2021. For reference purposes the VoR may still be accessed via the Supplemental Material section on this page.
ABSTRACT
In this paper, we present and evaluate a method for trajectory reconstruction from IMU signals generated when a person ”air writes” text with a finger worn IMU to make the resulting text as human-readable as possible. The vision is to provide a virtual ”sticky note” allowing people to digitally attach simple texts to locations. Thus, for example, we envision a person walking by someone’s locked office door and simply air writing, ”let me know when you are back”. The other person would then have, for example, their phone vibrate when they come into the office and would see the message on their screen. The problem that we address is how to extract from such ”air writing”, performed without visual feedback or a real surface to write, de-noised 2D trajectories that can be later displayed on a screen in a way that is well readable to humans. We describe the sensor and its signals, the trajectory extraction algorithm, and a user study that shows that we can achieve a high degree of readability.
Supplemental Material
Available for Download
Version of Record for "Finger Air Writing - Movement Reconstruction with Low-cost IMU Sensor" by Younas et al., MobiQuitous 2020 - 17th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous '20).
- C. Amma, M. Georgi, and T. Schultz. 2012. Airwriting: Hands-Free Mobile Text Input by Spotting and Continuous Recognition of 3d-Space Handwriting with Inertial Sensors. In 2012 16th International Symposium on Wearable Computers. 52–59.Google Scholar
- A. Dash, A. Sahu, R. Shringi, J. Gamboa, M. Z. Afzal, M. I. Malik, A. Dengel, and S. Ahmed. 2017. AirScript - Creating Documents in Air. In 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Vol. 01. 908–913.Google Scholar
- Z. Dong, U. C. Wejinya, and W. J. Li. 2010. An Optical-Tracking Calibration Method for MEMS-Based Digital Writing Instrument. IEEE Sensors Journal 10, 10 (2010), 1543–1551.Google ScholarCross Ref
- A. Kulshreshth, C. Zorn, and J. J. LaViola. 2013. Poster: Real-time markerless kinect based finger tracking and hand gesture recognition for HCI. In 2013 IEEE Symposium on 3D User Interfaces (3DUI). 187–188.Google ScholarCross Ref
- S. A. Ludwig and K. D. Burnham. 2018. Comparison of Euler Estimate using Extended Kalman Filter, Madgwick and Mahony on Quadcopter Flight Data. In 2018 International Conference on Unmanned Aircraft Systems (ICUAS). 1236–1241.Google Scholar
- S. O. H. Madgwick, A. J. L. Harrison, and R. Vaidyanathan. 2011. Estimation of IMU and MARG orientation using a gradient descent algorithm. In 2011 IEEE International Conference on Rehabilitation Robotics. 1–7.Google Scholar
- Pranav Mistry, Pattie Maes, and Liyan Chang. 2009. WUW - Wear Ur World: A Wearable Gestural Interface. In CHI ’09 Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA) (CHI EA ’09). Association for Computing Machinery, New York, NY, USA, 4111–4116. https://doi.org/10.1145/1520340.1520626Google ScholarDigital Library
- Lakshmi Prabha Nattamai Sekar, Alexander Santos, and Olga Beltramello. 2015. IMU Drift Reduction for Augmented Reality Applications. In Augmented and Virtual Reality, Lucio Tommaso De Paolis and Antonio Mongelli (Eds.). Springer International Publishing, Cham, 188–196.Google Scholar
- T. Pan, C. Kuo, H. Liu, and M. Hu. 2019. Handwriting Trajectory Reconstruction Using Low-Cost IMU. IEEE Transactions on Emerging Topics in Computational Intelligence 3, 3(2019), 261–270.Google Scholar
- T. Y. Pan, C. H. Kuo, and M. C. Hu. 2016. A noise reduction method for IMU and its application on handwriting trajectory reconstruction. In 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW). 1–6.Google ScholarCross Ref
- PyQT. 2012. PyQt Reference Guide. (2012). http://www.riverbankcomputing.com/static/Docs/PyQt4/html/index.htmlGoogle Scholar
- Maximilian Schrapel, Max-Ludwig Stadler, and Michael Rohs. 2018. Pentelligence: Combining Pen Tip Motion and Writing Sounds for Handwritten Digit Recognition. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3173574.3173705Google ScholarDigital Library
- Nabeel Siddiqui and Rosa H. M. Chan. 2020. Multimodal hand gesture recognition using single IMU and acoustic measurements at wrist. PLOS ONE 15, 1 (01 2020), 1–12. https://doi.org/10.1371/journal.pone.0227039Google Scholar
- Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2009. Brainy Hand: An Ear-Worn Hand Gesture Interaction Device. In CHI ’09 Extended Abstracts on Human Factors in Computing Systems (Boston, MA, USA) (CHI EA ’09). Association for Computing Machinery, New York, NY, USA, 4255–4260. https://doi.org/10.1145/1520340.1520649Google ScholarDigital Library
- Wenjin Tao, Ze-Hao Lai, Ming C. Leu, and Zhaozheng Yin. 2018. Worker Activity Recognition in Smart Manufacturing Using IMU and sEMG Signals with Convolutional Neural Networks. Procedia Manufacturing 26 (2018), 1159 – 1166. https://doi.org/10.1016/j.promfg.2018.07.15246th SME North American Manufacturing Research Conference, NAMRC 46, Texas, USA.Google ScholarCross Ref
- Huawei Tu, Xiangshi Ren, and Shumin Zhai. 2012. A Comparative Evaluation of Finger and Pen Stroke Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 1287–1296. https://doi.org/10.1145/2207676.2208584Google ScholarDigital Library
- Xiao J. Valenti RG, Dryanovski I. 2015. Keeping a Good Attitude: A Quaternion-Based Orientation Filter for IMUs and MARGs. Sensors 15, 8 (2015), 19302–19330. https://doi.org/10.3390/s150819302Google Scholar
- J. Wang and F. Chuang. 2012. An Accelerometer-Based Digital Pen With a Trajectory Recognition Algorithm for Handwritten Digit and Gesture Recognition. IEEE Transactions on Industrial Electronics 59, 7 (2012), 2998–3007.Google ScholarCross Ref
- F. Wittmann, O. Lambercy, R. R. Gonzenbach, M. A. van Raai, R. Höver, J. Held, M. L. Starkey, A. Curt, A. Luft, and R. Gassert. 2015. Assessment-driven arm therapy at home using an IMU-based virtual reality system. In 2015 IEEE International Conference on Rehabilitation Robotics (ICORR). 707–712.Google Scholar
- S. Won, W. Melek, and F. Golnaraghi. 2008. Position and orientation estimation using Kalman filtering and particle diltering with one IMU and one position sensor. In 2008 34th Annual Conference of IEEE Industrial Electronics. 3006–3010.Google Scholar
- S. Won, W. Melek, and F. Golnaraghi. 2008. Position and orientation estimation using Kalman filtering and particle diltering with one IMU and one position sensor. In 2008 34th Annual Conference of IEEE Industrial Electronics. 3006–3010.Google Scholar
- S. Won, W. Melek, and F. Golnaraghi. 2008. Position and orientation estimation using Kalman filtering and particle diltering with one IMU and one position sensor. In 2008 34th Annual Conference of IEEE Industrial Electronics. 3006–3010.Google Scholar
- Chao Xu, Parth H. Pathak, and Prasant Mohapatra. 2015. Finger-Writing with Smartwatch: A Case for Finger and Hand Gesture Recognition Using Smartwatch. In Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications (Santa Fe, New Mexico, USA) (HotMobile ’15). Association for Computing Machinery, New York, NY, USA, 9–14. https://doi.org/10.1145/2699343.2699350Google ScholarDigital Library
- D. Zhang, X. Wu, and C. Wang. 2017. Fine-Grained and Real-Time Gesture Recognition by Using IMU Sensors. In 2017 IEEE 23rd International Conference on Parallel and Distributed Systems (ICPADS). 747–754.Google Scholar
Recommendations
Optimal Placement of Multi-Channels sEMG Electrod for Finger Movement Classification
ICBBE '17: Proceedings of the 2017 4th International Conference on Biomedical and Bioinformatics EngineeringThis research aims to propose the optimal electrode positions for surface EMG by using a modern gesture control device called MYO armband. Seven healthy volunteers participated in this research. The sEMG signal was collected form three different ...
Model for unistroke writing time
CHI '01: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsUnistrokes are a viable form of text input in pen-based user interfaces. However, they are a very heterogeneous group of gestures the only common feature being that all are drawn with a single stroke. Several unistroke alphabets have been proposed ...
Touching character segmentation of Devanagari script
ICCCNT '16: Proceedings of the 7th International Conference on Computing Communication and Networking TechnologiesSegmentation of characters is one of the major step in OCR system. Devanagari script is a two dimensional form of symbol. It is very inconvenient to treat each form of character as a separate symbol because such combinations are very large in number. ...
Comments