Skip to main content
SpringerLink
Log in

New dissimilarity measures for image phylogeny reconstruction

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Image phylogeny is the problem of reconstructing the structure that represents the history of generation of semantically similar images (e.g., near-duplicate images). Typical image phylogeny approaches break the problem into two steps: (1) estimating the dissimilarity between each pair of images and (2) reconstructing the phylogeny structure. Given that the dissimilarity calculation directly impacts the phylogeny reconstruction, in this paper, we propose new approaches to the standard formulation of the dissimilarity measure employed in image phylogeny, aiming at improving the reconstruction of the tree structure that represents the generational relationships between semantically similar images. These new formulations exploit a different method of color adjustment, local gradients to estimate pixel differences and mutual information as a similarity measure. The results obtained with the proposed formulation remarkably outperform the existing counterparts in the literature, allowing a much better analysis of the kinship relationships in a set of images, allowing for more accurate deployment of phylogeny solutions to tackle traitor tracing, copyright enforcement and digital forensics problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. In our experiments, we have used the \(3 \times 3\) Sobel kernel. We performed some exploratory tests with other kernel sizes (e.g., \(3 \times 3\), \(5 \times 5\) and \(7 \times 7\)) but their performance was similar for the problem herein.

  2. A topology refers to the form of the trees in a forest. For instance, Fig. 1 depicts two different topologies for the set of images present on its left side.

  3. http://www.imagemagick.org/script/index.php.

  4. http://migre.me/vTYN7 (secure shortened link).

  5. For cases with \(n = 100\) images, the initial branching has \(n - 1 = 99\) edges. For creating a forest \({\mathcal {F}}\) where \(|{\mathcal {F}}| = 10\) trees, the number of total edges is \(n - |{\mathcal {F}}| = 100 - 10 = 90\).

  6. http://migre.me/vTYLt (secure shortened link).

References

  1. Battiti R (1994) Using mutual information for selecting features in supervised neural net learning. IEEE Trans Neural Netw (TNN) 5(4):537–550

    Article  Google Scholar 

  2. Bay H, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Elsevier Comput Vis Image Underst 110(3):346–359

    Article  Google Scholar 

  3. Bestagini P, Tagliasacchi M, Tubaro S (2016) Image phylogeny tree reconstruction based on region selection. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2059–2063

  4. Bramon R, Boada I, Bardera A, Rodriguez J, Feixas M, Puig J, Sbert M (2012) Multimodal data fusion based on mutual information. IEEE Trans Vis Comput Graph (TVCG) 18(9):1574–1587

    Article  Google Scholar 

  5. Brownlee KA (1965) Statistical theory and methodology in science and engineering. Wiley series in probability and mathematical statistics: applied probability and statistics. Wiley, New York

    MATH  Google Scholar 

  6. Costa F, Lameri S, Bestagini P, Dias Z, Rocha A, Tagliasacchi M, Tubaro S (2015) Phylogeny reconstruction for misaligned and compressed video sequences. In: IEEE international conference on image processing (ICIP), pp 301–305

  7. Costa F, Lameri S, Bestagini P, Dias Z, Tubaro S, Rocha A (2016) Hash-based frame selection for video phylogeny. In: IEEE international workshop on information forensics and security (WIFS)

  8. Costa F, Oikawa M, Dias Z, Goldenstein S, Rocha A (2014) Image phylogeny forests reconstruction. IEEE Trans Inf Forensics Secur (TIFS) 9(10):1533–1546

    Article  Google Scholar 

  9. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE computer society conference on computer vision and pattern recognition (CVPR), pp 886–893

  10. de Oliveira A, Ferrara P, De Rosa A, Piva A, Barni Mauro, Goldenstein S, Dias Z, Rocha A (2016) Multiple parenting phylogeny relationships in digital images. IEEE Trans Inf Forensics Secur(TIFS) 11(2):328–343

    Article  Google Scholar 

  11. Dias Z, Goldenstein S, Rocha A (2013) Exploring heuristic and optimum branching algorithms for image phylogeny. Elsevier J Vis Commun Image Represent 24:1124–1134

    Article  Google Scholar 

  12. Dias Z, Goldenstein S, Rocha A (2013) Large-scale image phylogeny: tracing back image ancestry relationships. IEEE Multimed 20:58–70

    Article  Google Scholar 

  13. Dias Z, Goldenstein S, Rocha A (2013) Toward image phylogeny forests: automatically recovering semantically similar image relationships. Elsevier Forensic Sci Int (FSI) 231:178–189

    Article  Google Scholar 

  14. Dias Z, Rocha A, Goldenstein S (2010) First steps toward image phylogeny. In: IEEE international workshop on information forensics and security (WIFS), pp 1–6

  15. Dias Z, Rocha A, Goldenstein S (2011) Video phylogeny: recovering near-duplicate video relationships. In: IEEE international workshop on information forensics and security (WIFS), pp 1–6

  16. Dias Z, Rocha A, Goldenstein S (2012) Image phylogeny by minimal spanning trees. IEEE Trans Inf Forensics Secur (TIFS) 7(2):774–788

    Article  Google Scholar 

  17. Edmonds J (1967) Optimum branchings. J Res Natl Inst Stand Technol 71B:48–50

    MATH  MathSciNet  Google Scholar 

  18. Fan Z, De Queiroz RL (2003) Identification of bitmap compression history: Jpeg detection and quantizer estimation. IEEE Trans Image Process 12(2):230–235

    Article  Google Scholar 

  19. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. ACM Commun 24(6):381–395

    Article  MathSciNet  Google Scholar 

  20. Gonzalez R, Woods R (2007) Digital image processing, 3rd edn. Prentice-Hall, New Jersey

    Google Scholar 

  21. Goshtasby AA (2012) Image registration: principles, tools and methods. advances in computer vision and pattern recognition, 1st edn. Springer, New York

    Book  MATH  Google Scholar 

  22. Hong C, Yu J, Tao D, Wang M (2015) Image-based three-dimensional human pose recovery by multiview locality-sensitive sparse retrieval. IEEE Trans Ind Electron 62(6):3742–3751

    Google Scholar 

  23. Joly A, Buisson O, Frélicot C (2007) Content-based copy retrieval using distortion-based probabilistic similarity search. IEEE Trans Multimed 9(2):293–306

    Article  Google Scholar 

  24. Kender JR, Hill ML, Natsev AP, Smith JR, Xie L (2010) Video genetics: a case study from youtube. In: International conference on multimedia, pp 1253–1258

  25. Kennedy L, Chang S-F (2008) Internet image archaeology: automatically tracing the manipulation history of photographs on the web. In: ACM international conference of multimedia, pp 349–358

  26. Lameri S, Bestagini P, Melloni A, Milani S, Rocha A, Tagliasacchi M, Tubaro S (2014) Who is my parent? Reconstructing video sequences from partially matching shots. In: IEEE international conference on image processing (ICIP), pp 5342–5346

  27. MacKinnon JG (1996) Numerical distribution functions for unit root and cointegration tests. J Appl Econom 11(6):601–618

    Article  Google Scholar 

  28. Maes F, Collignon A, Vandermeulen D, Marchal G, Suetens P (1997) Multimodality image registration by maximization of mutual information. IEEE Trans Med Imaging 16(2):187–198

    Article  Google Scholar 

  29. Mao J, Bulan O, Sharma G, Datta S (2009) Device temporal forensics: an information theoretic approach. In: IEEE international conference on image processing, pp 1485–1488

  30. Melloni A, Bestagini P, Milani S, Tagliasacchi M, Rocha A, Tubaro S (2014) Image phylogeny through dissimilarity metrics fusion. In: IEEE European workshop on visual information processing (EUVIP), pp 1–6

  31. Melloni A, Lameri S, Bestagini P, Tagliasacchi M, Tubaro S (2015) Near-duplicate detection and alignment for multi-view videos. In: IEEE international conference on image processing (ICIP), pp 1–4

  32. Milani S, Fontana M, Bestagini P, Tubaro S (2016) Phylogenetic analysis of near-duplicate images using processing age metrics. In: IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2054–2058

  33. Nucci M, Tagliasacchi M, Tubaro S (2013) A phylogenetic analysis of near-duplicate audio tracks. In: IEEE international workshop on multimedia, signal processing, pp 99–104

  34. Oikawa M, Dias Z, Rocha A, Goldenstein S (2016) Manifold learning and spectral clustering for image phylogeny forests. IEEE Trans Inf Forensics Secur 11(1):5–18

    Article  Google Scholar 

  35. Oliveira A, Ferrara P, De Rosa A, Piva A, Barni M, Goldenstein S, Dias Z, Rocha A (2014) Multiple parenting identification in image phylogeny. In: IEEE international conference on image processing (ICIP), pp 5347–5351

  36. Oliveira A, Ferrara P, De Rosa A, Piva A, Barni M, Goldenstein S, Dias Z, Rocha A (2016) Multiple parenting phylogeny relationships in digital images. IEEE Trans Inf Forensics and Secur (TIFS) 11(2):328–343

    Article  Google Scholar 

  37. Reinhard E, Ashikhmin M, Gooch B, Shirley P (2001) Color transfer between images. IEEE Comput Graph Appl 21:34–41

    Article  Google Scholar 

  38. De Rosa A, Uccheddu F, Costanzo A, Piva A, Barni M (2010) Exploring image dependencies: a new challenge in image forensics. SPIE Med Forensics Secur 7541(2):1–12

    Google Scholar 

  39. Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: an efficient alternative to SIFT or SURF. In: IEEE international conference on computer vision (ICCV), pp 2564–2571

  40. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(379–423):623–656

    Article  MATH  MathSciNet  Google Scholar 

  41. Sobel I, Feldman G (1968) A \(3\times 3\) isotropic gradient operator for image processing. In: Artificial Project in a talk at the Stanford, pp 271–272

  42. Tapia JE, Perez CA (2013) Gender classification based on fusion of different spatial scale features selected by mutual information from histogram of LBP, intensity, and shape. IEEE Trans Inf Forensics Secury (TIFS) 8(3):488–499

    Article  Google Scholar 

  43. Viola P, Wells WM (1997) Alignment by maximization of mutual information. Int J Comput Vis 24:137–154

    Article  Google Scholar 

  44. Wilcoxon F (1945) Individual comparisons by ranking methods. Biom Bull 1(6):80–83

    Article  Google Scholar 

  45. Yu J, Rui Y, Chen B (2014) Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans Multimed 16(1):159–168

    Article  Google Scholar 

  46. Yu J, Yang X, Gao F, Tao D (2016) Deep multimodal distance metric learning using click constraints for image ranking. IEEE Trans Cybern. doi:10.1109/TCYB.2016.2591583

    Google Scholar 

  47. Zitová B, Flusser J (2003) Image registration methods: a survey. Image Vis Comput 21:977–1000

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the Brazilian Coordination for Higher Education and Personnel (CAPES) through the CAPES DeepEyes Project, the São Paulo Research Foundation (Grants #2013/05815-2 and the DéjàVu Project #2015/19222-9), Microsoft Research and the European Union through the REWIND (REVerse engineering of audio-VIsual coNtent Data) project for the financial support. Finally, it is important to mention that this material is also based on research sponsored by DARPA and Air Force Research Laboratory (AFRL) under agreement number FA8750-16-2-0173. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA and Air Force Research Laboratory (AFRL) or the U.S. Government.

Author information

Authors and Affiliations

  1. Institute of Computing, University of Campinas, Campinas, Brazil

    Filipe Costa, Alberto Oliveira, Zanoni Dias, Siome Goldenstein & Anderson Rocha

  2. Università degli Studi di Firenze, Florence, Italy

    Pasquale Ferrara

Authors
  1. Filipe Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alberto Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pasquale Ferrara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zanoni Dias
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Siome Goldenstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anderson Rocha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Filipe Costa or Anderson Rocha.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 92 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Costa, F., Oliveira, A., Ferrara, P. et al. New dissimilarity measures for image phylogeny reconstruction. Pattern Anal Applic 20, 1289–1305 (2017). https://doi.org/10.1007/s10044-017-0616-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-017-0616-9

Keywords

Search

Navigation