skip to main content
10.1145/3634814.3634845acmotherconferencesArticle/Chapter ViewAbstractPublication PagesasseConference Proceedingsconference-collections
research-article

Dimension-Wise Feature Selection of Deep Learning Models for In-Air Signature Time Series Analysis Based on Shapley Values

Published: 26 March 2024 Publication History

Abstract

This paper performs a comprehensive evaluation of Smartwatch in-air signature classification based on multiple deep learning models. We leverage the Shapley value in dimension-wise feature selection to provide the in-air signature community with the most and least dominant dimension regarding the accuracy of in-air signature classification. Our experiment results highlight InceptionTime as the top-performing model, achieving an accuracy of 97.73%. Through our Shapley Value analysis, among all the sensors embedded in the Smartwatch, we find that the y dimension of the gyroscope and the z dimension of the gyroscope contribute the most to classification accuracy with 12.57% and 12.51% respectively, while the x dimension of the accelerometer produces the least contribution with 8.71%.

References

[1]
Abien Fred Agarap. 2018. Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375 (2018).
[2]
Gonzalo Bailador, Carmen Sanchez-Avila, Javier Guerra-Casanova, and Alberto de Santos Sierra. 2011. Analysis of pattern recognition techniques for in-air signature biometrics. Pattern Recognition 44, 10-11 (2011), 2468–2478.
[3]
Mustafa Gokce Baydogan and George Runger. 2016. Time series representation and similarity based on local autopatterns. Data Mining and Knowledge Discovery 30 (2016), 476–509.
[4]
Donald J Berndt and James Clifford. 1994. Using dynamic time warping to find patterns in time series. In Proceedings of the 3rd international conference on knowledge discovery and data mining. 359–370.
[5]
Attaullah Buriro, Bruno Crispo, Filippo Delfrari, and Konrad Wrona. 2016. Hold and sign: A novel behavioral biometrics for smartphone user authentication. In 2016 IEEE security and privacy workshops (SPW). IEEE, 276–285.
[6]
Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning 20 (1995), 273–297.
[7]
Hoang Anh Dau, Eamonn Keogh, Kaveh Kamgar, Chin-Chia Michael Yeh, Yan Zhu, Shaghayegh Gharghabi, Chotirat Ann Ratanamahatana, Yanping, Bing Hu, Nurjahan Begum, Anthony Bagnall, Abdullah Mueen, Gustavo Batista, and Hexagon-ML. 2018. The UCR Time Series Classification Archive. https://www.cs.ucr.edu/ eamonn/time_series_data_2018/.
[8]
Evelyn Fix and Joseph Lawson Hodges. 1989. Discriminatory analysis. Nonparametric discrimination: Consistency properties. International Statistical Review/Revue Internationale de Statistique 57, 3 (1989), 238–247.
[9]
Claudio Gallicchio and Simone Scardapane. 2020. Deep Randomized Neural Networks. CoRR abs/2002.12287 (2020). arXiv:2002.12287https://arxiv.org/abs/2002.12287
[10]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision. 1026–1034.
[11]
Tin Kam Ho. 1995. Random decision forests. In Proceedings of 3rd international conference on document analysis and recognition, Vol. 1. IEEE, 278–282.
[12]
Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arxiv:1502.03167 [cs.LG]
[13]
Hassan Ismail Fawaz, Germain Forestier, Jonathan Weber, Lhassane Idoumghar, and Pierre-Alain Muller. 2019. Deep learning for time series classification: a review. Data mining and knowledge discovery 33, 4 (2019), 917–963.
[14]
Hassan Ismail Fawaz, Benjamin Lucas, Germain Forestier, Charlotte Pelletier, Daniel F Schmidt, Jonathan Weber, Geoffrey I Webb, Lhassane Idoumghar, Pierre-Alain Muller, and François Petitjean. 2020. Inceptiontime: Finding alexnet for time series classification. Data Mining and Knowledge Discovery 34, 6 (2020), 1936–1962.
[15]
Gen Li and Hiroyuki Sato. 2022. Sensing In-Air Signature Motions Using Smartwatch: A High-Precision Approach of Behavioral Authentication. IEEE Access 10 (2022), 57865–57879. https://doi.org/10.1109/ACCESS.2022.3177905
[16]
Jameel Malik, Ahmed Elhayek, Suparna Guha, Sheraz Ahmed, Amna Gillani, and Didier Stricker. 2020. DeepAirSig: End-to-End Deep Learning Based in-Air Signature Verification. IEEE Access 8 (2020), 195832–195843. https://doi.org/10.1109/ACCESS.2020.3033848
[17]
Abdulhalık Oğuz and Ömer Faruk Ertuğrul. 2022. Human identification based on accelerometer sensors obtained by mobile phone data. Biomedical signal processing and control 77 (2022), 103847.
[18]
L. Rabiner and B. Juang. 1986. An introduction to hidden Markov models. IEEE ASSP Magazine 3, 1 (1986), 4–16. https://doi.org/10.1109/MASSP.1986.1165342
[19]
Frank Rosenblatt. 1957. The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory.
[20]
Benedek Rozemberczki, Lauren Watson, Péter Bayer, Hao-Tsung Yang, Olivér Kiss, Sebastian Nilsson, and Rik Sarkar. 2022. The Shapley Value in Machine Learning. arxiv:2202.05594 [cs.LG]
[21]
Baljit Singh Saini, Parminder Singh, Anand Nayyar, Navdeep Kaur, Kamaljit Singh Bhatia, Shaker El-Sappagh, and Jong-Wan Hu. 2020. A three-step authentication model for mobile phone user using keystroke dynamics. IEEE Access 8 (2020), 125909–125922.
[22]
Mohammad Saleem and B Kovari. 2019. Survey of signature verification databases. In International Multidisciplinary Scientific Conference.
[23]
Joan Serra, Santiago Pascual, and Alexandros Karatzoglou. 2018. Towards a Universal Neural Network Encoder for Time Series. In CCIA. 120–129.
[24]
Lloyd S Shapley 1953. A value for n-person games. (1953).
[25]
Erik Strumbelj and Igor Kononenko. 2010. An efficient explanation of individual classifications using game theory. The Journal of Machine Learning Research 11 (2010), 1–18.
[26]
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2017. Instance Normalization: The Missing Ingredient for Fast Stylization. arxiv:1607.08022 [cs.CV]
[27]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. arxiv:1706.03762 [cs.CL]
[28]
Shixuan Wang, Jiabin Yuan, and Jing Wen. 2019. Adaptive phone orientation method for continuous authentication based on mobile motion sensors. In 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 1623–1627.
[29]
Zhiguang Wang, Weizhong Yan, and Tim Oates. 2017. Time series classification from scratch with deep neural networks: A strong baseline. In 2017 International joint conference on neural networks (IJCNN). IEEE, 1578–1585.
[30]
Kevin Yeo, Ooi Shih Yin, Pang Ying Han, and Wee Kuok Kwee. 2015. Real time mobile application of in-air signature with Fast Dynamic Time Warping (FastDTW). In 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA). IEEE, 315–320.
[31]
Bendong Zhao, Huanzhang Lu, Shangfeng Chen, Junliang Liu, and Dongya Wu. 2017. Convolutional neural networks for time series classification. Journal of Systems Engineering and Electronics 28, 1 (2017), 162–169.
[32]
Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. 2014. Time series classification using multi-channels deep convolutional neural networks. In International conference on web-age information management. Springer, 298–310.
[33]
Yi Zheng, Qi Liu, Enhong Chen, Yong Ge, and J Leon Zhao. 2016. Exploiting multi-channels deep convolutional neural networks for multivariate time series classification. Frontiers of Computer Science 10 (2016), 96–112.
[34]
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. 2016. Learning deep features for discriminative localization. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2921–2929.

Index Terms

  1. Dimension-Wise Feature Selection of Deep Learning Models for In-Air Signature Time Series Analysis Based on Shapley Values

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ASSE '23: Proceedings of the 2023 4th Asia Service Sciences and Software Engineering Conference
    October 2023
    267 pages
    ISBN:9798400708534
    DOI:10.1145/3634814
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 26 March 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Classification
    2. Encoder
    3. FCN
    4. Feature Selection
    5. Game Theory
    6. In-Air Signature
    7. Inception-Time
    8. MC-DCNN
    9. MLP
    10. Machine Learning
    11. ResNet
    12. Sensors
    13. Shapley Value
    14. Time Series
    15. Time-CNN

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Funding Sources

    • KAKENHI (Grant-in-Aid for JSPS Fellows) 21J21087
    • University of Tokyo, Graduate School of Engineering, Department of Computing and Communication Systems, SEUT scholarship

    Conference

    ASSE 2023

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 25
      Total Downloads
    • Downloads (Last 12 months)25
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format.

    HTML Format

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media