Skip to main content
Springer Nature Link
Log in

How to Compare Image Watermarking Algorithms

  • Chapter
Transactions on Data Hiding and Multimedia Security IV

Part of the book series: Lecture Notes in Computer Science ((TDHMS,volume 5510))

  • 331 Accesses

Abstract

In recent years, comparing image watermarking algorithms using the results of experiments with a handful of images has become a common practice. The images are randomly selected from a well chosen database and so it can be argued that they cover the important aspects of visual information such as texture or smoothness. However, the number of images used in the experiment remains a crucial factor in providing confidence in the results of experiment. By choosing this number to be ‘large’ one hopes that the comparison results are reliable. In this paper, our aim is to provide a systematic method of determining this number using a statistical approach based on hypothesis testing and power analysis.

We consider two algorithms and seek to verify the claim that one has superior performance with respect to some property. We start with finding a measure that represents the ‘goodness’ of the property of interest, follow with determining a suitable hypothesis test and the number of images and, finally, with interpreting the results of the experiment. We show how the use of a statistical framework can allow us not only to draw conclusions based on a sound methodology but also to provide a confidence level for these conclusions. We give a concrete application of the approach to the problem of comparing two spread spectrum watermarking algorithms with respect to their robustness to JPEG2000 compression. We discuss the intricacies of the choices at various stages of this approach, and their effects on the results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Chen, T.P.C., Chen, T.: A framework for optimal blind watermark detection. In: ACM Multimedia 2001 Workshop on Multimedia and Security, Ottawa, Ontario, Canada (October 2001)

    Google Scholar 

  2. Mukherjee, D., Maitra, S., Acton, S.: Spatial domain digital watermarking of multimedia objects for buyer authentication. IEEE Transactions on Multimedia 6, 1–15 (February 2004)

    Article  Google Scholar 

  3. Armeni, S., Christodoulakis, D., Kostopoulos, I., Stamatiou, Y., Xenos, M.: A transparent watermarking method for color images. In: First IEEE Balkan Conf. on Signal Processing, Communications, Circuits, and Systems, Istanbul, Turkey (June 2000)

    Google Scholar 

  4. Qiao, L., Cox, I.: Using perceptual models to improve fidelity and provide invariance to valumetric scaling for quantization index modulation watermarking. In: Proc. IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing (ICASSP 2005), vol. 2, pp. 1–4 (March 2005)

    Google Scholar 

  5. Fridrich, J., Goljan, M.: Comparing robustness of watermarking techniques. In: Wong, P.W., Delp, E.J. (eds.) Security and Watermarking of Multimedia Contents, vol. 3657, pp. 214–225. SPIE, San Jose (January 1999)

    Chapter  Google Scholar 

  6. Dittmann, J., Megías, D., Lang, A., Herrera-Joancomartí, J.: Theoretical framework for a practical evaluation and comparison of audio watermarking schemes in the triangle of robustness, transparency and capacity. In: Shi, Y.Q. (ed.) Transactions on Data Hiding and Multimedia Security I. LNCS, vol. 4300, pp. 1–40. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  7. Kutter, M., Petitcolas, F.A.P.: A fair benchmark for image watermarking systems. In: Wong, P.W., Delp, E.J. (eds.) Security and Watermarking of Multimedia Contents, vol. 3657, pp. 226–239. SPIE, San Jose (January 1999)

    Chapter  Google Scholar 

  8. Kim, H.C., Ogunley, H., Guitart, O., Delp, E.J.: The watermark evaluation testbed. In: Wong, P., Delp, E.J. (eds.) Proc. SPIE Int’l Conf. on Security and Watermarking of Multimedia Contents, San Jose, California, USA (January 2004)

    Google Scholar 

  9. ISO/IEC JTC 1/SC 29/WG 1: ISO/IEC FCD 15444-1: Information technology - jpeg2000 image coding system: Core coding system [WG 1 N 1646] (March 2000)

    Google Scholar 

  10. Su, J.K., Eggers, J.J., Girod, B.: Analysis of digital watermarks subjected to optimum linear filtering and additive noise. Signal Processing, Special Issue on Information Theoretic Issues in Digital Watermarking 8 (2001)

    Google Scholar 

  11. Adelsbach, A., Katzenbeisser, S., Sadeghi, A.-R.: A computational model for watermark robustness. In: Camenisch, J.L., Collberg, C.S., Johnson, N.F., Sallee, P. (eds.) IH 2006. LNCS, vol. 4437, pp. 145–160. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  12. Pereira, S., Voloshynovskiy, S., Madueo, M., Marchand-Maillet, S., Pun, T.: Second generation benchmarking and application oriented evaluation. In: Moskowitz, I.S. (ed.) IH 2001. LNCS, vol. 2137, pp. 340–353. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  13. Solachidis, V., Tefas, A., Nikolaidis, N., Tsekeridou, S., Nikolaidis, A., Pitas, I.: A benchmarking protocol for watermarking methods. In: Proc. IEEE International Conference on Image Processing, Thessaloniki, Greece, vol. 3, pp. 1023–1026 (October 2001)

    Google Scholar 

  14. Macq, B., Dittmann, J., Delp, E.: Benchmarking of image watermarking algorithms for digital rights management. Proc. of the IEEE 92(6), 971–984 (2004)

    Article  Google Scholar 

  15. Kim, H.C., Delp, E.J.: A reliability engineering approach to digital watermark evaluation. In: Wong, P., Delp, E.J. (eds.) Proc. SPIE Int’l Conf. on Security and Watermarking of Multimedia Contents (January 2006)

    Google Scholar 

  16. Fridrich, J.: Combining low-frequency and spread spectrum watermarking. In: Proc. SPIE International Symposium on Optical Science, Engineering, and Instrumentation (1998)

    Google Scholar 

  17. Barni, M., Bartolini, F., Cappellini, V., Lippi, A., Piva, A.: A DWT-based technique for spatio-frequency masking of digital signatures. In: SPIE International Conf. on Security and Watermarking of Multimedia Contents, San Jose, CA, vol. 3657, pp. 31–39 (January 1999)

    Google Scholar 

  18. van Schyndel, R.G., Tirkel, A.Z., Osborne, C.F.: A digital watermark. In: International Conference on Image Processing, Austin, Texas, USA, vol. 2, pp. 86–90 (1994)

    Google Scholar 

  19. Bender, W., Gruhl, D., Morimoto, N., Lu, A.: Techniques for data hiding. IBM Systems Journal 35(3-4), 313–336 (1996)

    Article  Google Scholar 

  20. Cox, I., Kilian, J., Leighton, T., Shamoon, T.: Secure spread spectrum watermarking for multimedia. IEEE Trans. Image Processing 6(12), 1673–1687 (1997)

    Article  Google Scholar 

  21. Voloshynovskiy, S., Pereira, S., Iquise, V., Pun, T.: Attack modelling: Towards a second generation watermarking benchmark (2001) Signal Processing, Special Issue on Information Theoretic Issues in Digital Watermarking (2001)

    Google Scholar 

  22. Cox, I., Miller, M., Bloom, J.: Digital watermarking. Morgan Kaufmann Publishers Inc., San Francisco (2002)

    Google Scholar 

  23. Zhang, X., Wandell, B.: A spatial extension of CIELAB for digital color image reproduction. In: Proc. Soc. Inform. Display 1996 Digest, San Diego, pp. 731–734 (1996)

    Google Scholar 

  24. Bickel, P.J., Doksum, K.A.: 4. In: Mathematical Statistics: Basic Ideas and Selected Topics, Holden-Day, Inc., Oakland (1977)

    Google Scholar 

  25. Lee, W.: Experimental Design and Analysis. W. H. Freeman and Company, San Fransisco (1975)

    Google Scholar 

  26. Hollander, M., Wolfe, D.A.: Nonparametric Statistical Methods. John Wiley & Sons, Inc., New York (1973)

    MATH  Google Scholar 

  27. The gimp-savvy photo archive, http://gimp-savvy.com/PHOTO-ARCHIVE/index.html (last access: 5 May 2005)

  28. Sharma, G., Wu, W., Dalal, E.: The CIEDE2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research and Application 30(1), 21–30 (2005)

    Article  Google Scholar 

  29. Morovic, J., Sun, P.: Visual differences in colour reproduction and their colorimetric correlates. In: IS&T/SID 10th Color Imaging Conference, pp. 292–297 (2002)

    Google Scholar 

  30. Shapiro, S.S., Wilk, M.: An analysis of variance test for normality (complete samples). Biometrika 52(3/4), 591–611 (1965)

    Article  MathSciNet  MATH  Google Scholar 

  31. Shaffer, J.P.: Multiple hypothesis testing. Annual Review of Psychology 46, 561–576 (1995)

    Article  Google Scholar 

  32. Nadenau, M.J.: Integration of Human Colour Vision Models into High Quality Image Compression. PhD thesis, Ecole Polytechnique Federale de Lausanne, Switzerland (2000)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of IT and CS, University of Wollongong, Australia

    Angela Piper

  2. Dept. of Computer Science, University of Calgary, Canada

    Reihaneh Safavi-Naini

Authors
  1. Angela Piper
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reihaneh Safavi-Naini
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. New Jersey Institute of Technology, University Heights, 07102-1982, Newark, NJ, USA

    Yun Q. Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Piper, A., Safavi-Naini, R. (2009). How to Compare Image Watermarking Algorithms. In: Shi, Y.Q. (eds) Transactions on Data Hiding and Multimedia Security IV. Lecture Notes in Computer Science, vol 5510. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01757-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01757-5_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01756-8

  • Online ISBN: 978-3-642-01757-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics