Skip to main content
Springer Nature Link
Log in

A Study on the Usability of Handwriting Assistant for Smartphone’s Lock Screen

  • Conference paper
  • First Online:
Design, Operation and Evaluation of Mobile Communications (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14052))

Included in the following conference series:

  • 607 Accesses

Abstract

Advances in the development of devices with handwritten input, and the emergence of deep learning have led to the rapid development of handwriting recognition applications, but the methods and implications of handwritten interaction on the Lock Screen of the smartphone, taking into account usability, are still not sufficiently considered in existing studies. In this paper, we propose a novel method for the quick processing of handwritten input and delivering results without explicitly starting the presumed application associated with an action. Handwritten input is processed either with or without requiring authorization, depending on the sensitivity of the implied action. The method comprises the initial handwritten input processing to locate and extract textual information, then entity extraction from the recognized text, and device context identification. The extracted entities and the current device context are analyzed to identify the action imposed by the user, to process the input, and to obtain and deliver the results in a single or a few steps. The proposed method is suitable for integration with the first screen of the device, also known as the Lock Screen, but it can also be utilized in applications based on handwriting input. The user study demonstrates that the proposed method accelerates the time to completion (TTC) of the most common actions by 20% compared to traditional input when all the necessary steps are required to be explicitly performed by the user.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andriotis, P., Oikonomou, G., Mylonas, A., Tryfonas, T.: A study on usability and security features of the android pattern lock screen. Inf. Comput. Secur. 24(1), 53–72 (2016)

    Article  Google Scholar 

  2. Banovic, N., Brant, C., Mankoff, J., Dey, A.: ProactiveTasks: the short of mobile device use sessions. In: Proceedings of the 16th International Conference on Human-Computer Interaction with Mobile Devices & Services, pp. 243–252 (2014)

    Google Scholar 

  3. Bott, J.N., Laviola Jr, J.J.: The WOZ recognizer: a wizard of Oz sketch recognition system. ACM Trans. Interact. Intell. Syst. 5, 1–38 (2015)

    Google Scholar 

  4. Cooper, R., Ali, S., Bi, C.: Extracting information from short messages. In: Montoyo, A., Muńoz, R., Métais, E. (eds.) NLDB 2005. LNCS, vol. 3513, pp. 388–391. Springer, Heidelberg (2005). https://doi.org/10.1007/11428817_44

    Chapter  Google Scholar 

  5. Degtyarenko, I., et al.: Hierarchical recurrent neural network for handwritten strokes classification. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2865–2869 (2021)

    Google Scholar 

  6. Forsberg, A.S., Bragdon, A., Jr., J.J.L., Raghupathy, S., Zeleznik, R.C.: An empirical study in pen-centric user interfaces: diagramming. In: Alvarado, C., Cani, M.P. (eds.) Eurographics Workshop on Sketch-Based Interfaces and Modeling. The Eurographics Association (2008)

    Google Scholar 

  7. Frinken, V., Uchida, S.: Deep BLSTM neural networks for unconstrained continuous handwritten text recognition. In: 2015 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 911–915. IEEE (2015)

    Google Scholar 

  8. Graves, A., Fernández, S., Gomez, F., Schmidhuber, J.: Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In: Proceedings of the 23rd International Conference on Machine learning, pp. 369–376 (2006)

    Google Scholar 

  9. Gu, J., Lee, G.: Towards more direct text editing with handwriting interfaces. Int. J. Hum.-Comput. Interact. 39(1), 233–248 (2023)

    Article  Google Scholar 

  10. Ha, D., Eck, D.: A neural representation of sketch drawings. In: International Conference on Learning Representations (2018)

    Google Scholar 

  11. Harbach, M., De Luca, A., Egelman, S.: The anatomy of smartphone unlocking: a field study of android lock screens. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 4806–4817 (2016)

    Google Scholar 

  12. Heitmayer, M.: “It’s Like Being Gone For A Second”: using subjective evidence-based ethnography to understand locked smartphone use among young adults. In: Proceedings of the 23rd International Conference on Mobile Human-Computer Interaction. MobileHCI 2021, Association for Computing Machinery, New York, NY, USA (2021). https://doi.org/10.1145/3447526.3472026

  13. Hsu, Y.-H., Chen, C.-H.: Usability study on the user interface design of tablet note-taking applications. In: Stephanidis, C., Antona, M., Ntoa, S. (eds.) HCII 2021. CCIS, vol. 1419, pp. 423–430. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78635-9_55

    Chapter  Google Scholar 

  14. Ihara, A.S., Nakajima, K., Kake, A., Ishimaru, K., Osugi, K., Naruse, Y.: Advantage of handwriting over typing on learning words: evidence from an N400 event-related potential index. Front. Hum. Neurosci. 15, 679191 (2021)

    Article  Google Scholar 

  15. Kim, S.H., Kwon, M.S., Kim, D.H., Lee, D.H., Hwang, S.t.: Apparatus and method for executing functions related to handwritten user input on lock screen. Google Patents, 05 March 2015. https://patents.google.com/patent/US20150062041A1/en, US Patent 0062041

  16. Koushki, M.M., Obada-Obieh, B., Huh, J.H., Beznosov, K.: Is implicit authentication on smartphones really popular? On android users’ perception of “smart lock for android”. In: 22nd International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 1–17 (2020)

    Google Scholar 

  17. Read, J.C.: A study of the usability of handwriting recognition for text entry by children. Interact. Comput. 19(1), 57–69 (2006)

    Article  Google Scholar 

  18. Roschelle, J., Tatar, D., Chaudhury, S.R., Dimitriadis, Y., Patton, C., DiGiano, C.: Ink, improvisation, and interactive engagement: learning with tablets. Computer 40(9), 42–48 (2007)

    Article  Google Scholar 

  19. Samsung: Use Screen off memo on your Galaxy Note. https://www.samsung.com/us/support/answer/ANS00080345/. Accessed 07 Feb 2023

  20. Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Ortega-Garcia, J.: DeepSign: deep on-line signature verification. IEEE Trans. Biometrics Behav. Identity Sci. 3(2), 229–239 (2021)

    Article  Google Scholar 

  21. Truong, K.N., Shihipar, T., Wigdor, D.J.: Slide to X: unlocking the potential of smartphone unlocking. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3635–3644 (2014)

    Google Scholar 

  22. Wohllebe, A., Adler, M.R., Podruzsik, S.: Influence of design elements of mobile push notifications on mobile app user interactions. Int. J. Interact. Mob. Technol. 15(15), 35–46 (2021)

    Article  Google Scholar 

  23. Zaytsev, V., Zhelezniakov, D., Hirilishena, L.: Electronic device for performing operation on basis of handwriting input, and method for operating same. World Intellectual Property Organization, 02 September 2021. https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2021172648. WO2021172648

  24. Zhelezniakov, D., Cherneha, A., Zaytsev, V., Ignatova, T., Radyvonenko, O., Yakovchuk, O.: Evaluating new requirements to pen-centric intelligent user interface based on end-to-end mathematical expressions recognition. In: Proceedings of the ACM International Conference on Intelligent User Interfaces, pp. 212–220 (2020)

    Google Scholar 

  25. Zhelezniakov, D., Zaytsev, V., Radyvonenko, O.: Acceleration of online recognition of 2D sequences using deep bidirectional LSTM and dynamic programming. In: Rojas, I., Joya, G., Catala, A. (eds.) IWANN 2019. LNCS, vol. 11507, pp. 438–449. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20518-8_37

    Chapter  Google Scholar 

  26. Zhelezniakov, D., Zaytsev, V., Radyvonenko, O., Yakishyn, Y.: InteractivePaper: minimalism in document editing UI through the handwriting prism. In: The Adjunct Publication of the 32nd Annual ACM Symposium on User Interface Software and Technology, pp. 13–15 (2019)

    Google Scholar 

  27. Zhong, Q., Guo, G., Zhi, J.: Chinese handwriting while driving: effects of handwritten box size on in-vehicle information systems usability and driver distraction. Traffic Inj. Prev. 24, 1–6 (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Samsung R &D Institute, Kyiv, Ukraine

    Viktor Zaytsev, Dmytro Zhelezniakov, Anastasiia Cherneha & Olga Radyvonenko

  2. Taras Shevchenko National University of Kyiv, Kyiv, Ukraine

    Dmytro Zhelezniakov

Authors
  1. Viktor Zaytsev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dmytro Zhelezniakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anastasiia Cherneha
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olga Radyvonenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Viktor Zaytsev .

Editor information

Editors and Affiliations

  1. University of Central Florida, Orlando, FL, USA

    Gavriel Salvendy

  2. University of West Florida, Pensacola, FL, USA

    June Wei

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zaytsev, V., Zhelezniakov, D., Cherneha, A., Radyvonenko, O. (2023). A Study on the Usability of Handwriting Assistant for Smartphone’s Lock Screen. In: Salvendy, G., Wei, J. (eds) Design, Operation and Evaluation of Mobile Communications . HCII 2023. Lecture Notes in Computer Science, vol 14052. Springer, Cham. https://doi.org/10.1007/978-3-031-35921-7_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35921-7_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35920-0

  • Online ISBN: 978-3-031-35921-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics