Abstract
The verification performance of biometric systems is normally evaluated using the receiver operating characteristic (ROC) or detection error trade-off (DET) curve. We propose two new ideas for statistical evaluation of biometric systems based on these data. The first is a new way to normalize match score distributions. A normalized match score, \(\hat{t}\), is calculated as a function of the angle from a representation of (FMR, FNMR) values in polar coordinates from some center. This has the advantage that it does not produce counterintuitive results for systems with unusual DET performance. Secondly, building on this normalization we develop a methodology to calculate an average DET curve. Each biometric system is represented in terms of \(\hat{t}\) to allow genuine and impostor distributions to be combined, and an average DET is then calulated from these new distributions. We then show that this method is equivalent to direct averaging of DET data along each angle from the center. This procedure is then applied to data from a study of human matchers of facial images.
This work is supported by NSERC Canada (Dr. Adler), and by NSF grant CNS-0325640 (Dr. Schuckers) which is cooperatively funded by the National Science Foundation and the United States Department of Homeland Security
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Adler, A., Maclean, J.: Performance comparison of human and automatic face recognition. In: Biometrics Consortium Conference 2004, Washington, DC, USA, Septmeber 20-22 (2004)
Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognition 7, 1145–1159 (1997)
Burton, A.M., Miller, P., Bruce, V., Hancock, P.J.B., Henderson, Z.: Human and automatic face recognition: a comparison across image formats. Vision Research 41, 3185–3195 (2001)
Drummond, C., Holte, R.C.: What ROC Curves Can’t Do (and Cost Curves Can). In: Proc. 1st Workshop ROC Analysis in AI:ROCAI, pp. 19–26 (2004)
Fawcett, T.: ROC graphs: Notes and practical considerations for data mining researchers, Technical Report HPL-2003-4. HP Labs (2003)
Green, D.M., Swets, J.A.: Signal Detection Theory and Psychophysics. JohnWiley & Sons, New York (1966)
Golfarelli, M., Maio, D., Maltoni, D.: On the Error-Reject Trade-Off in Biometric Verification Systems. IEEE Trans. Pattern Anal. Machine Intel. 19, 786–796 (1997)
Hancock, P.J.B., Bruce, V., Burton, M.A.: A comparison of two computerbased face identification systems with human perceptions of faces. Vision Research 38, 2277–2288 (1998)
Hanley, J.A., McNeil, B.J.: The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143, 29–36 (1982)
Hernández-Orallo, J., Ferri, C., Lachiche, N., Flach, P.A. (eds.): ROC Analysis in Artificial Intelligence, 1st Int. Workshop, ROCAI-2004, Valencia, Spain (2004)
Jain, A.K., Nandakumar, K., Ross, A.: Score Normalization in Multimodal Biometric Systems. Pattern Recognition (2005) (in press)
Karduan, J., Karduan, O.: Comparative diagnostic performance of three radiological procedures for the detection of lumbar disk herniation. Methods Inform. Med. 29, 12–22 (1990)
Macskassy, S., Provost, F.: Confidence Bands for ROC Curves: Methods and an Empirical Study. In: Proc. 1st Workshop ROC Analysis in AI:ROCAI, pp. 61–70 (2004)
NIST: Face Recognition Vendor test (2002), http://frvt.org/frvt2002
NIST: Fingerprint Vendor Technology Evaluation, FpVTE (2003), http://fpvte.nist.gov/
NIST: NIST Special Database 18: Mugshot Identification Database (MID), http://www.nist.gov/srd/nistsd18.htm
Provost, F.J., Fawcett, T., Kohavi, R.: The case against accuracy estimation for comparing induction algorithms. In: Proc. 15th Int. Conf. Machine Learning, pp. 445–453 (1998)
Rukhin, A., Grother, P., Phillips, P.J., Newton, E.: Dependence characteristics of face recognition algorithms. In: Proc. Int. Conf Pattern Recog., vol. 16, pp. 36–39 (2002)
Zhou, X.-H., McClish, D.K., Obuchowski, N.A.: Statistical Methods in Diagnostic Medicine. John W. Wiley & Sons, Chichester (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Adler, A., Schuckers, M.E. (2005). Calculation of a Composite DET Curve. In: Kanade, T., Jain, A., Ratha, N.K. (eds) Audio- and Video-Based Biometric Person Authentication. AVBPA 2005. Lecture Notes in Computer Science, vol 3546. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11527923_90
Download citation
DOI: https://doi.org/10.1007/11527923_90
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-27887-0
Online ISBN: 978-3-540-31638-1
eBook Packages: Computer ScienceComputer Science (R0)