Salma Mandi
ABSTRACT
Keystroke or typing dynamics represent two key facets - rhythm corresponds to spectral-domain characteristics and timing corresponds to time-domain behavior, which are created when a person types. The presence of inherent time-domain and frequency-domain characteristics in smartphone keyboard interactions motivate us to perform a comparative analysis of time-domain and frequency-domain features for emotion detection. We design, and develop an Android-based data collection application, which collects keyboard interaction logs and emotion self-reports (happy, sad, stressed, relaxed) from 18 subjects in a 3-week in-the-wild study. For the time-domain analysis, we extract a set of time-domain features and construct Random Forest-based personalized model; whereas for the spectral-domain analysis, first transform the interaction details into frequency-domain using DFT (Discrete Fourier Transform) and then extract a set of spectral-domain features to construct a personalized model for emotion detection. The empirical analysis from the study reveals that the time-domain models return superior classification performance (average AUCROC 72%) than the frequency-domain models (average 67%). It also signifies the importance of several time-domain and frequency-domain features as a strong discriminator of emotion states.
- Bokai Cao, Lei Zheng, Chenwei Zhang, Philip S Yu, Andrea Piscitello, John Zulueta, Olu Ajilore, Kelly Ryan, and Alex D Leow. 2017. Deepmood: modeling mobile phone typing dynamics for mood detection. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 747--755.Google Scholar
Digital Library
- Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321--357.Google ScholarCross Ref
- M. Garrido, K. K. Parhi, and J. Grajal. 2009. A Pipelined FFT Architecture for Real-Valued Signals. IEEE Transactions on Circuits and Systems I: Regular Papers 56, 12 (2009), 2634--2643.Google ScholarCross Ref
- Surjya Ghosh, Niloy Ganguly, Bivas Mitra, and Pradipta De. 2017. Evaluating effectiveness of smartphone typing as an indicator of user emotion. In 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII). IEEE, 146--151.Google ScholarCross Ref
- Surjya Ghosh, Niloy Ganguly, Bivas Mitra, and Pradipta De. 2017. TapSense: Combining Self-report Patterns and Typing Characteristics for Smartphone Based Emotion Detection. In Proceedings of the ACM MobileHCI.Google ScholarDigital Library
- S. Ghosh, N. Ganguly, B. Mitra, and P. De. 2019. Designing An Experience Sampling Method for Smartphone based Emotion Detection. IEEE Transactions on Affective Computing (2019), 1--1. Google ScholarDigital Library
- Md Rakibul Islam, Md Kauser Ahmmed, and Minhaz F. Zibran. 2019. MarValous: Machine Learning Based Detection of Emotions in the Valence-Arousal Space in Software Engineering Text. In Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing (Limassol, Cyprus) (SAC '19). Association for Computing Machinery, New York, NY, USA, 1786--1793. Google ScholarDigital Library
- Huy Viet Le, Sven Mayer, Patrick Bader, and Niels Henze. 2018. Fingers' Range and Comfortable Area for One-Handed Smartphone Interaction Beyond the Touchscreen. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--12.Google ScholarDigital Library
- Sven Mayer, Huy Viet Le, Markus Funk, and Niels Henze. 2019. Finding the Sweet Spot: Analyzing Unrestricted Touchscreen Interaction In-the-Wild. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces. 171--179.Google ScholarDigital Library
- James A Russell. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6 (1980), 1161--1178.Google ScholarCross Ref
- Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.Google ScholarDigital Library
Index Terms
- Emotion detection from smartphone keyboard interactions: role of temporal vs spectral features
Recommendations
AffectPro: Towards Constructing Affective Profile Combining Smartphone Typing Interaction and Emotion Self-reporting Pattern
ICMI '22: Proceedings of the 2022 International Conference on Multimodal InteractionThe ubiquity of smartphones and the widespread usage of text entry by soft keyboard in different instant messaging applications (e.g., WhatsApp, FB messenger) have opened the possibilities of inferring emotions from longitudinal typing data. To build ...
Emotion Detection through Smartphone's Accelerometer and Gyroscope Sensors
UMAP '21: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and PersonalizationEmotion recognition is essential for assessing human emotional states and predicting user behavior to provide appropriate and personalized feedback. The wide range of Smartphones with accelerometers, microphones, GPSs, gyroscopes, and more motivate ...
Effectiveness of Deep Neural Network Model in Typing-based Emotion Detection on Smartphones
MobiCom '18: Proceedings of the 24th Annual International Conference on Mobile Computing and NetworkingTyping characteristics on smartphones can provide clues for emotion detection. Collecting large volumes of typing data is also easy on smartphones. This motivates the use of Deep Neural Network (DNN) to determine emotion states from smartphone typing. ...
Comments