A novel video-based system for in-air signature verification

https://doi.org/10.1016/j.compeleceng.2016.11.010Get rights and content

Highlights

  • Develop a novel video-base in-air signature verification mode.

  • Propose an improved DTW algorithm based on the characteristic of in-air signature.

  • Demonstrate the validity of FFT algorithm in the verification of continuous track signature.

  • Design a DTW-FFT fusion algorithm based on Gaussian probability distribution.

Abstract

At present, the common online signature verification systems adopt the contacted model of signing on a tablet or a phone screen, which have a great dependence on the device. Considering that the development of biometrics tends to the long distance and non-contact mode, a novel video-based system for in-air signature verification is proposed. First, the fingertip tracking is used for generating unique signature trajectory from the in-air signing videos. In combination of contrapuntally improved dynamic time warping (DTW) method, the Fast Fourier Transform (FFT) and the analysis on the signature length, a Gaussian distribution-based fusion algorithm is proposed for in-air signature verification. Finally, comprehensive experiments are conducted on a self-built database consisting of 560 signatures, then a false rejection rate (FRR) of 2.86% and a false acceptance rate (FAR) of 1.90% are achieved with an average matching time of only 24 ms, which have demonstrated the effectiveness of our proposed system.

Introduction

With the advent of the information age, the information revolution made a huge impact on the traditional work and life styles. In view of the critical role of identity authentication in the online payment and settlement, the information security has been suffering severely test and the electronic identity authentication has become an important part of the information age. Meanwhile, with the great progress of biometric authentication technology, biometrics as an authentication password has gradually attains to the requirements of automated, real-time and accurate, thus it has attracted the widespread attention in the domain of pattern recognition. Commonly used biometrics includes fingerprint, face, voice, signature, gait, vein etc. As ancient and generally accepted biological characteristics among them, signature has been widely studied and applied to many occasions which need identity authentication.

In general, signature verification can be divided into off-line (static) and online (dynamic) signature verification. Off-line signature verification is a technique for the identification of 2-D signature images written on a document, which only consists of the shape information. While online signature verification technology is based on the dynamic features in signing on the digital tablet or PAD, such as coordinates and pen pressure at each point along the signature trajectory [1], which makes the online signatures more distinct and robust, as well as more difficult to be forged because of the diversity of the features and the integrity of the signature information.

The overall process of online signature verification includes data acquisition, preprocessing, feature extraction and matching. Among them, feature extraction plays a very important role and existed strategies can be divided into two categories, parameter-based and function-based. The advantage of the former is more reliable, easier access features with smaller size and less computation. The most commonly used parameters include the position, velocity, acceleration, pen strokes etc. [2]. While function-based features utilize timing signals to describe the signatures, such as orbit, angular velocity, pressure and orientation, which are considered to be more capable to reflect the content of the signature [3]. It has been demonstrated that speeds on X-coordinates and Y-coordinates are the most efficient features by close observation and analysis on stability observation [4]. Parodi et al. obtained the optimal feature combination by the consistency factor [5]. Recently, Fayyaz, et al. also achieved good performance by utilizing a sparsity autoencoder to train features from signatures [6].

As for matching process, most of the classic pattern recognition algorithms have been adopted for signature verification [7]. Among them, the dynamic time warping algorithm (DTW) is the most popular scheme [8], and shows its superiority on similarity measurement when the length of the two samples are not matched. To further increase the accuracy, a team from Turkey Sabanci University improved the principal component analysis (PCA) based on DTW method in First International Signature Verification Competition (SVC2004), which achieved the best performance with 2.84% and 2.89% EER in task 1 and task 2 [9], [10]. Meanwhile, there are some studies on improving the DTW performance by selecting special points to narrow the area of dynamic warping [11]. In addition, other methods such as Hidden Markov Model (HMM) [12], Support Vector Machine (SVM) [13], Fourier Transform (FT) [14], Neural Network (NN) [15], have also been successfully applied for the signature authentication system.

The above researches are mainly based on the 2-D signature information, which is obtained from the plane only. With customers’ increasing demands for usability, flexibility, friendliness and security, 2-D signature is gradually beginning to reveal its deficiencies, especially its security will face significant challenges with emergency of automatic signature machine (autopen). Aimed to the above problems, 3-D non-contact authentication method appears and attracts more attention for its non-imitativeness and friendliness. But the data acquisition of signature trajectory in 3-D space is far more difficult than that on 2-D digital tablet, which makes it becomes the most challenging part for the in-air online signature verification.

In 3-D data acquisition for signature verification, high speed cameras and hand held devices were firstly introduced, and the result of 3.5% FAR with 3.6% FRR was obtained in the 96 sets of data, by means of the commercial signature authentication engine from JAPAN CyberSIGN company [16]. Next, the infrared and RGB cameras were also utilized to get the 3-D trajectory of signature, by combining with the hand shape features to recognize the English words from A to J, it achieved the result of 2% FAR when FRR is zero [17]. Recently, the smart phone's acceleration sensor is explored to obtain 3-D trajectory of in-air signature, by converting it to the 2-D signature data and then using SVM as a classifier, the paper achieved EER of 0.8% in the self-built database [18]. In addition, since the sign's gesture is changing with time, a template updating algorithm was proposed to improve the system performance with finally 4% EER [19]. After comparing with HMM, Bayes classifier, DTW and other algorithms for the in-air signature authentication, Bailador et al. demonstrated that the DTW algorithm achieves the best performance [20]. As a new wearable device, the Google glasses were used to track finger trajectories in the first person vision and the signature verification was conducted by the DTW algorithm successfully [21].

Although the in-air signature has drawn more attention in recent years, the research is still at a beginning stage. And most research work relies on mobile phones or tablet, as well as other external handheld devices for data acquisition and system communication. Aiming at the low cost, low equipment dependence for practical in-air online signature verification system, we put forward a novel authentication mode for non-contact video-based finger signature action. The main contributions of this paper are as follows: a) Develop a novel video-base in-air signature verification mode; b) Propose an improved DTW algorithm based on the characteristic of in-air signature; c) Demonstrate the validity of FFT algorithm in the verification of continuous track signature; d) Design a DTW-FFT fusion algorithm based on Gaussian probability distribution.

The rest of this paper is organized as follows. Section II describes our proposed method in detail. Section III shows experiments and analysis on the results. Finally, we discuss the conclusions and future work in section IV.

Section snippets

Method

The implementation framework of video-based in-air signature verification system is similar to that of common online signature verification system, both of them include registration and verification steps. However, because this paper aims at the in-air signature trajectory acquired through video analysis, so the steps of data acquisition and pre-processing are totally different from that of online signature verification system. What's more, we also design a new feature extraction and matching

Database

Since there is no video signature database available online, we construct one database by ourselves. A 100 fps high-speed camera was utilized to record the fast movement of the fingertip in the video, and then the improved TLD tracking algorithm was used to track and generate the writing trajectory. It is demonstrated that the improved TLD algorithm has strong robustness to the upward, downward, leftward and rightward sloping of the fingertip under the condition of varied intensity, complex

Conclusions and future work

Through in-depth analysis and research, we have developed a novel video-based mode for in-air signature verification aiming at current issues of online signature verification and in line with the growth trends of biometrics. To improve the effect of verification, the proposed algorithm has integrated the time and space information of in-air signature by DTW and FFT. Compared to other algorithms that more than ten training samples are needed, our proposed scheme only requires five genuine

Acknowledgments

This work is supported by the National Natural Science Foundation of China (Grant Nos. 61573151, 61105019, and 61503141), by Guangdong Natural Science Foundation (Grant No. 2016A030313468), Featured Innovation Project in Colleges and Universities of Guangdong Province (Grant No. 2015KTSCX007), and by the Science and Technology Program of Guangzhou (Grant No. 201510010088).

Yuxun Fang received the B.E. degree in automation from South China University of Technology, Guangzhou, China, in 2016, and is currently working toward the M.S. degree at the South China University of Technology, Guangzhou, China. His research interests include biometrics identification, pattern recognition, and computer vision.

References (25)

  • M. Fayyaz et al.

    Online signature verification based on feature representation

  • Z. Zhang et al.

    A survey of on-line signature verification

    Biometric recognition

    (2011)
  • Cited by (31)

    • In-air hand gesture signature using transfer learning and its forgery attack

      2021, Applied Soft Computing
      Citation Excerpt :

      Table 10 addresses the verification performance comparison of forgery attacks with the other existing methods. The compared methods include approaches that utilize the smartphone’s accelerometer signals as input [42,43], approaches that derive signature trajectory as input data captured by a depth camera [40,41,45] as well as our approach which is the only work that utilizes 3D imagery as input data. The results show that our approach able to obtain a decent low error rate of 0.68% random forgery attacks as with other state-of-arts.

    • Dynamic signatures: A review of dynamic feature variation and forensic methodology

      2018, Forensic Science International
      Citation Excerpt :

      We must point out that the study was of very limited scope and that results should be considered with great care. Many types of digitizing devices exist, including gloves, mobile devices, camera-based devices and dedicated signature tablets [102–109]. Every digitizing mode has its own particularities and a different set of recorded measurements.

    View all citing articles on Scopus

    Yuxun Fang received the B.E. degree in automation from South China University of Technology, Guangzhou, China, in 2016, and is currently working toward the M.S. degree at the South China University of Technology, Guangzhou, China. His research interests include biometrics identification, pattern recognition, and computer vision.

    Wenxiong Kang received the M.S. degree from Northwestern Polytechnical University, Xi'an, China, in 2003, and the Ph.D. degree from South China University of Technology, Guangzhou, China, in 2009. He is currently an Associate Professor with the School of Automation Science and Engineering, South China University of Technology. His research interests include biometrics identification, image processing, pattern recognition, and computer vision.

    Qiuxia Wu received the Ph.D. degree from South China University of Technology, Guangzhou, China, in 2012. From October 2009 to October 2011, she was a Visiting Student with The University of Sydney, Australia. She is currently an Assistant Professor with the School of Software Engineering, South China University of Technology. Her research interests include biometrics identification and pattern recognition.

    Lei Tang received the B.E. degree in automation from Qingdao University of Science and Technology, Qingdao, China, in 2014, and is currently working toward the M.S. degree at the South China University of Technology, Guangzhou, China. His research interests include online signature verification and pattern recognition.

    View full text