A novel video-based system for in-air signature verification
Graphical abstract
Introduction
With the advent of the information age, the information revolution made a huge impact on the traditional work and life styles. In view of the critical role of identity authentication in the online payment and settlement, the information security has been suffering severely test and the electronic identity authentication has become an important part of the information age. Meanwhile, with the great progress of biometric authentication technology, biometrics as an authentication password has gradually attains to the requirements of automated, real-time and accurate, thus it has attracted the widespread attention in the domain of pattern recognition. Commonly used biometrics includes fingerprint, face, voice, signature, gait, vein etc. As ancient and generally accepted biological characteristics among them, signature has been widely studied and applied to many occasions which need identity authentication.
In general, signature verification can be divided into off-line (static) and online (dynamic) signature verification. Off-line signature verification is a technique for the identification of 2-D signature images written on a document, which only consists of the shape information. While online signature verification technology is based on the dynamic features in signing on the digital tablet or PAD, such as coordinates and pen pressure at each point along the signature trajectory [1], which makes the online signatures more distinct and robust, as well as more difficult to be forged because of the diversity of the features and the integrity of the signature information.
The overall process of online signature verification includes data acquisition, preprocessing, feature extraction and matching. Among them, feature extraction plays a very important role and existed strategies can be divided into two categories, parameter-based and function-based. The advantage of the former is more reliable, easier access features with smaller size and less computation. The most commonly used parameters include the position, velocity, acceleration, pen strokes etc. [2]. While function-based features utilize timing signals to describe the signatures, such as orbit, angular velocity, pressure and orientation, which are considered to be more capable to reflect the content of the signature [3]. It has been demonstrated that speeds on X-coordinates and Y-coordinates are the most efficient features by close observation and analysis on stability observation [4]. Parodi et al. obtained the optimal feature combination by the consistency factor [5]. Recently, Fayyaz, et al. also achieved good performance by utilizing a sparsity autoencoder to train features from signatures [6].
As for matching process, most of the classic pattern recognition algorithms have been adopted for signature verification [7]. Among them, the dynamic time warping algorithm (DTW) is the most popular scheme [8], and shows its superiority on similarity measurement when the length of the two samples are not matched. To further increase the accuracy, a team from Turkey Sabanci University improved the principal component analysis (PCA) based on DTW method in First International Signature Verification Competition (SVC2004), which achieved the best performance with 2.84% and 2.89% EER in task 1 and task 2 [9], [10]. Meanwhile, there are some studies on improving the DTW performance by selecting special points to narrow the area of dynamic warping [11]. In addition, other methods such as Hidden Markov Model (HMM) [12], Support Vector Machine (SVM) [13], Fourier Transform (FT) [14], Neural Network (NN) [15], have also been successfully applied for the signature authentication system.
The above researches are mainly based on the 2-D signature information, which is obtained from the plane only. With customers’ increasing demands for usability, flexibility, friendliness and security, 2-D signature is gradually beginning to reveal its deficiencies, especially its security will face significant challenges with emergency of automatic signature machine (autopen). Aimed to the above problems, 3-D non-contact authentication method appears and attracts more attention for its non-imitativeness and friendliness. But the data acquisition of signature trajectory in 3-D space is far more difficult than that on 2-D digital tablet, which makes it becomes the most challenging part for the in-air online signature verification.
In 3-D data acquisition for signature verification, high speed cameras and hand held devices were firstly introduced, and the result of 3.5% FAR with 3.6% FRR was obtained in the 96 sets of data, by means of the commercial signature authentication engine from JAPAN CyberSIGN company [16]. Next, the infrared and RGB cameras were also utilized to get the 3-D trajectory of signature, by combining with the hand shape features to recognize the English words from A to J, it achieved the result of 2% FAR when FRR is zero [17]. Recently, the smart phone's acceleration sensor is explored to obtain 3-D trajectory of in-air signature, by converting it to the 2-D signature data and then using SVM as a classifier, the paper achieved EER of 0.8% in the self-built database [18]. In addition, since the sign's gesture is changing with time, a template updating algorithm was proposed to improve the system performance with finally 4% EER [19]. After comparing with HMM, Bayes classifier, DTW and other algorithms for the in-air signature authentication, Bailador et al. demonstrated that the DTW algorithm achieves the best performance [20]. As a new wearable device, the Google glasses were used to track finger trajectories in the first person vision and the signature verification was conducted by the DTW algorithm successfully [21].
Although the in-air signature has drawn more attention in recent years, the research is still at a beginning stage. And most research work relies on mobile phones or tablet, as well as other external handheld devices for data acquisition and system communication. Aiming at the low cost, low equipment dependence for practical in-air online signature verification system, we put forward a novel authentication mode for non-contact video-based finger signature action. The main contributions of this paper are as follows: a) Develop a novel video-base in-air signature verification mode; b) Propose an improved DTW algorithm based on the characteristic of in-air signature; c) Demonstrate the validity of FFT algorithm in the verification of continuous track signature; d) Design a DTW-FFT fusion algorithm based on Gaussian probability distribution.
The rest of this paper is organized as follows. Section II describes our proposed method in detail. Section III shows experiments and analysis on the results. Finally, we discuss the conclusions and future work in section IV.
Section snippets
Method
The implementation framework of video-based in-air signature verification system is similar to that of common online signature verification system, both of them include registration and verification steps. However, because this paper aims at the in-air signature trajectory acquired through video analysis, so the steps of data acquisition and pre-processing are totally different from that of online signature verification system. What's more, we also design a new feature extraction and matching
Database
Since there is no video signature database available online, we construct one database by ourselves. A 100 fps high-speed camera was utilized to record the fast movement of the fingertip in the video, and then the improved TLD tracking algorithm was used to track and generate the writing trajectory. It is demonstrated that the improved TLD algorithm has strong robustness to the upward, downward, leftward and rightward sloping of the fingertip under the condition of varied intensity, complex
Conclusions and future work
Through in-depth analysis and research, we have developed a novel video-based mode for in-air signature verification aiming at current issues of online signature verification and in line with the growth trends of biometrics. To improve the effect of verification, the proposed algorithm has integrated the time and space information of in-air signature by DTW and FFT. Compared to other algorithms that more than ten training samples are needed, our proposed scheme only requires five genuine
Acknowledgments
This work is supported by the National Natural Science Foundation of China (Grant Nos. 61573151, 61105019, and 61503141), by Guangdong Natural Science Foundation (Grant No. 2016A030313468), Featured Innovation Project in Colleges and Universities of Guangdong Province (Grant No. 2015KTSCX007), and by the Science and Technology Program of Guangzhou (Grant No. 201510010088).
Yuxun Fang received the B.E. degree in automation from South China University of Technology, Guangzhou, China, in 2016, and is currently working toward the M.S. degree at the South China University of Technology, Guangzhou, China. His research interests include biometrics identification, pattern recognition, and computer vision.
References (25)
- et al.
Legendre polynomials based feature extraction for online signature verification. Consistency analysis of feature combinations
Pattern Recogn
(2014) - et al.
Identity authentication using improved online signature verification method
Pattern Recogn Lett
(2005) - et al.
Online signature verification using a new extreme points warping technique
Pattern Recogn Lett
(2003) - et al.
Signature verification using multiple neural classifiers
Pattern Recogn
(1997) - et al.
Analysis of pattern recognition techniques for in-air signature biometrics
Pattern Recogn.
(2011) - et al.
HMM-based on-line signature verification: feature extraction and signature modeling
Pattern Recogn Lett
(2007) - et al.
Sensor interoperability and fusion in signature verification: a case study using tablet pc
Advances in biometric person authentication
(2005) - et al.
Reliable online human signature verification systems
IEEE Trans Pattern Anal Mach Intell
(1996) Stroke-based time warping for signature verification
- H. Lei and V. Govindaraju, "A comparative study on the consistency of features in on-line signature verification,"...
Online signature verification based on feature representation
A survey of on-line signature verification
Biometric recognition
Cited by (31)
In-air hand gesture signature using transfer learning and its forgery attack
2021, Applied Soft ComputingCitation Excerpt :Table 10 addresses the verification performance comparison of forgery attacks with the other existing methods. The compared methods include approaches that utilize the smartphone’s accelerometer signals as input [42,43], approaches that derive signature trajectory as input data captured by a depth camera [40,41,45] as well as our approach which is the only work that utilizes 3D imagery as input data. The results show that our approach able to obtain a decent low error rate of 0.68% random forgery attacks as with other state-of-arts.
Dynamic signatures: A review of dynamic feature variation and forensic methodology
2018, Forensic Science InternationalCitation Excerpt :We must point out that the study was of very limited scope and that results should be considered with great care. Many types of digitizing devices exist, including gloves, mobile devices, camera-based devices and dedicated signature tablets [102–109]. Every digitizing mode has its own particularities and a different set of recorded measurements.
Enhanced In-air Signature Verification via Hand Skeleton Tracking to Defeat Robot-level Replays
2023, ACM International Conference Proceeding SeriesmmHSV: In-Air Handwritten Signature Verification via Millimeter-Wave Radar
2023, ACM Transactions on Internet of ThingsSmartwatch In-Air Signature Time Sequence Three-Dimensional Static Restoration Classification Based on Multiple Convolutional Neural Networks
2023, Applied Sciences (Switzerland)
Yuxun Fang received the B.E. degree in automation from South China University of Technology, Guangzhou, China, in 2016, and is currently working toward the M.S. degree at the South China University of Technology, Guangzhou, China. His research interests include biometrics identification, pattern recognition, and computer vision.
Wenxiong Kang received the M.S. degree from Northwestern Polytechnical University, Xi'an, China, in 2003, and the Ph.D. degree from South China University of Technology, Guangzhou, China, in 2009. He is currently an Associate Professor with the School of Automation Science and Engineering, South China University of Technology. His research interests include biometrics identification, image processing, pattern recognition, and computer vision.
Qiuxia Wu received the Ph.D. degree from South China University of Technology, Guangzhou, China, in 2012. From October 2009 to October 2011, she was a Visiting Student with The University of Sydney, Australia. She is currently an Assistant Professor with the School of Software Engineering, South China University of Technology. Her research interests include biometrics identification and pattern recognition.
Lei Tang received the B.E. degree in automation from Qingdao University of Science and Technology, Qingdao, China, in 2014, and is currently working toward the M.S. degree at the South China University of Technology, Guangzhou, China. His research interests include online signature verification and pattern recognition.