Exploring heuristic and optimum branching algorithms for image phylogeny

Highlights

• We explore one heuristic algorithm for finding the evolution history of images.

• We explore one optimum branching algorithm for the same task.

• Applications are tracking image broadcasting or the chain of image distribution.

• 350,000 test cases show the effectiveness of the proposed methods.

• Our solution finds the ancestry connections in a set and the original document.

Abstract

Currently, multimedia objects can be easily created, stored, (re)-transmitted, and edited for good or bad. In this sense, there has been an increasing interest in finding the structure of temporal evolution within a set of documents and how documents are related to one another overtime. This process, also known in the literature as Multimedia Phylogeny, aims at finding the phylogeny tree(s) that best explains the creation process of a set of near-duplicate documents (e.g., images/videos) and their ancestry relationships. Solutions to this problem have direct applications in forensics, security, copyright enforcement, news tracking services and other areas. In this paper, we explore one heuristic and one optimum branching algorithm for reconstructing the evolutionary tree associated with a set of image documents. This can be useful for aiding experts to track the source of child pornography image broadcasting or the chain of image distribution in time, for instance. We compare the algorithms with the state-of-the-art solution considering 350,000 test cases and discuss advantages and disadvantages of each one in a real scenario.

Introduction

Nowadays, popular images and videos spread out at a rapid pace on the web through blogs, news sites, and social media, described by the popular expression to go viral. It is straightforward to find exact duplicates among available media, but many types of media objects, such as images and videos, can suffer small modifications during the redistribution that can change them without interfering on their semantic meaning. Some example include A/D or D/A conversions, (de)-coding, noise due to transmission, and small editing/corrections as brightness adjustments, and cropping. These are called near-duplicates, and are part of an active research area [1], [2], [3], [4], [5].

While most of these changes are natural and not necessarily harmful, sometimes the distribution itself may cause copyright infringement or even be a criminal action [6]. In some situations, the spreading pattern of an image or video can help companies to understand demographics and effectiveness of an ad campaign or a product.

These scenarios motivated the advent of a new research subfield called Multimedia Phylogeny [7], [8], [9], [10], [11], with the objective of investigating the history and evolutionary process of digital objects.

Multimedia Phylogeny goes beyond the detection of near-duplicate objects which only seeks to determine if a set of objects are near-duplicates of one another. In Multimedia Phylogeny, we are interested in finding the structure of modifications of a set of multimedia objects, including their causal and ancestry relationships, source of modifications (root or patient zero) of a set of related documents, reconstructing the order and transformations that originally created the near-duplicate set [7], [8], [9], [10], [11]. Fig. 1 illustrates one case for images. The underlying question in Multimedia Phylogeny is how we find the relationships between each pair of images to point out which document generated another one across time and which transformation parameters were used.

Solutions to this problem have applications in different areas, such as the ones listed bellow:

• Security: the modification graph of a set of documents might provide information of suspects’ behavior, and point out the directions of online content distribution.

• Forensics: better results might be achieved if the forensic analysis is performed in the original document instead of on a near-duplicate [6], [12]. In addition, forensic experts might focus their attention on individuals associated with the redistribution of content closer to the root of the tree as they are more likely to be the ones who created the content at first place.

• Copyright enforcement: traitor tracing without the requirement of active source control solutions such as the ones based on either watermarking or fingerprinting approaches.

• News tracking services: Multimedia Phylogeny might be a valuable tool for mining applications. The near-duplicate relationships can feed news tracking services with key elements for determining the opinion formation process across time and space [7], [8].

• Information retrieval: content-based retrieval systems might be developed to allow the display of similar photographs coming from different photographers without any metadata analysis.

Current approaches state that there are two steps necessary to reconstruct the phylogeny tree. The first one consists of calculating a dissimilarity matrix whose entries compare every two documents (e.g., images) in a set. For the second step, this matrix is then used to reconstruct the tree. In this sense, Rosa et al. [8] make the decision at every pair of images on what should be the right direction, while Dias et al. [9], [11] have a global method to construct a minimum spanning tree.

Following the same methodology discussed in previous works, in this paper we explore one heuristic and one optimum branching algorithm for reconstructing the evolutionary tree associated with a set of image documents. The heuristic solution is inspired on Prim’s classic minimum spanning tree algorithm [13] while the optimum branching algorithm is based on the classical Chu–Liu, Bock and Edmonds algorithm [14], [15], [16]. We compare both methods to the state-of-the-art Oriented Kruskal algorithm [11] using the same dissimilarity matrices for image near-duplicate sets and discuss the pros and cons of each approach.