Abstract
Most biometric authentication algorithms make use of a similarity score that defines how similar two templates are according to a threshold and the accuracy of the results are expressed in terms of a False Reject Rate (FRR) or False Accept Rate (FAR) that is estimated using the training data set. A confidence interval is assigned to any claim of accuracy with 90% being commonly assumed for biometric-based authentication systems. However, these confidence intervals may not be as accurate as is presumed. In this paper, we report the results of experiments measuring the performance of the widely-used subset bootstrap approach to estimating the confidence interval of FAR. We find that the coverage of the FAR confidence intervals estimated by the subset bootstrap approach is reduced by the dependence between two similarities when they come from two individual pairs shared with a common individual. This is because subset bootstrap requires the independence of different subsets. To deal with this, we present a second-level partition to the similarity score set between different individuals, producing what we call a subset false accept rate (SFAR) bootstrap estimation. The experimental results show that the proposed procedures greatly increase the coverage of the FAR confidence intervals.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Shen, W., Surette, M., Khanna, R.: Evaluation of Automated Biometrics-Based Identification and Verification Systems. Proceedings of the IEEE 85(9), 1464–1478 (1997)
Golfarelli, M., Maio, D., Maltoni, D.: On the Error-Reject Trade-Off in Biometric Verification Systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(7) (July 1997)
Politis, D.N.: Computer-intensive methods in statistical analysis. IEEE Signal Processing Magazine 15(1), 39–55 (1998)
Wayman, J.L.: Error rate equations for the general biometric system. IEEE Robotics & Automation Magazine 6(1), 35–48 (1999)
Phillips, P.J., Martin, A., Wilson, C.L., Przybocki, M.: An Introduction to Evaluating Biometric Systems. Computer 33(2), 56–63 (2000)
Bolle, R.M., Pankanti, S., Ratha, N.K.: Evaluation techniques for biometrics-based authentication systems (FRR). In: 15th International Conference on Pattern Recognition, Proceedings, September 2000, vol. 2(3-7), pp. 831–837 (2000)
Dahel, S.K., Xiao, Q.: Accuracy performance analysis of multimodal biometrics. In: Information Assurance Workshop, IEEE Systems, Man and Cybernetics Society, June 18-20, pp. 170–173 (2003)
Bolle, R.M., Ratha, N.K., Pankanti, S.: IBM Thomas J. Watson Research Center. In: Proceedings of the 17th International Conference on An Evaluation of Error Confidence Interval Estimation Methods, August 23-26, vol. 3, pp. 103–106 (2004)
Bolle, R.M., Ratha, N.K., Pankanti, S.: Error analysis of pattern recognition systems—the subsets bootstrap. Computer Vision and Image Understanding 93(1), 1–33 (2004)
Schmid, N.A., O’Sullivan, J.A.: Performance Prediction Methodology for Biometric Systems Using a Large Deviations Approach. IEEE Transactions on Signal Processing 52(10) (October 20, 2004)
Gamassi, M., Lazzaroni, M., Misino, M., Piuri, V., Sana, D., Scotti, F.: Quality Assessment of Biometric Systems: A Comprehensive Perspective Based on Accuracy and Performance Measurement. IEEE Transactions on Instrumentation and Measurement 54(4) (August 2005)
Snelick, R., Uludag, U., Mink, A., Indovina, M., Jain, A.: Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(3), 450–455 (2005)
Hube, J.P.: Using Biometric Verification to Estimate Identification Performance. In: Biometric Consortium Conference, 2006 Biometrics Symposium: Special Session on Research, pp. 1–6 (2006)
Poh, N., Bengio, S.: Estimating the Confidence Interval of Expected Performance Curve in Biometric Authentication using Joint Bootstrap. In: IEEE International Conference on Acoustics, Speech and Signal Processing, April 15-20, vol. 2, pp. II-137 – II-140 (2007)
Li, W., Zhang, D., Xu, Z.Q.: Palmprint Recognition by Fourier Transform. Journal of Software 5(13), 879–886 (2002)
Li, W., You, J., Zhang, D.: A Texture-based Approach to Palmprint Retrieval for Personal Identification. Accepted by IEEE Trans. on Multimedia (2003)
Kong, W.K., Zhang, D., Li, W.: Palmprint Feature Extraction Using 2-D Gabor Filters. Pattern Recognition 36(10), 2339–2347 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, R., Tang, D., Li, W., Zhang, D. (2009). Second-Level Partition for Estimating FAR Confidence Intervals in Biometric Systems. In: Jiang, X., Petkov, N. (eds) Computer Analysis of Images and Patterns. CAIP 2009. Lecture Notes in Computer Science, vol 5702. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03767-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-03767-2_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03766-5
Online ISBN: 978-3-642-03767-2
eBook Packages: Computer ScienceComputer Science (R0)