A critical literature survey and prospects on tampering and anomaly detection in image data

https://doi.org/10.1016/j.asoc.2020.106727Get rights and content

Highlights

  • A rigorous survey contemplating the state-of-the-art literature on computer aided tampered image and anomaly detection.

  • Image tampering detection represents a solution to improve enhancement falsify.

  • Datasets used in recent work focusing on anomaly detection for hyperspectral imagery.

  • Datasets have characteristics aimed at applied in the detection of altered images.

  • Seam carving imagery tampering within the machine learning context.

Abstract

Concernings related to image security have increased in the last years. One of the main reasons relies on the replacement of conventional photography to digital images, once the development of new technologies for image processing, as much as it has helped in the evolution of many new techniques in forensic studies, it also provided tools for image tampering. In this context, many companies and researchers devoted many efforts towards methods for detecting such tampered images, mostly aided by autonomous intelligent systems. Therefore, this work focuses on introducing a rigorous survey contemplating the state-of-the-art literature on computer-aided tampered image detection using machine learning techniques, as well as evolutionary computation, neural networks, fuzzy logic, Bayesian reasoning, among others. Besides, it also contemplates anomaly detection methods in the context of images due to the intrinsic relation between anomalies and tampering. Moreover, it aims at recent and in-depth researches relevant to the context of image tampering detection, performing a survey over more than 100 works related to the subject, spanning across different themes related to image tampering detection. Finally, a critical analysis is performed over this comprehensive compilation of literature, yielding some research opportunities and discussing some challenges in an attempt to align future efforts of the community with the niches and gaps remarked in this exciting field.

Introduction

In the last decade, digital images have thoroughly replaced conventional photographs. With that advancement, the usage of digital images in a computational forensics field has become more apparent. The significant development in digital image processing, as much as it has helped in the evolution of many new techniques in forensic studies, has also made image tampering easy. In this sense, Image security has become a critical issue in all field that makes use of digital images. Tampered images have long been contents of forensic studies; for example, images of criminals, images of crime scenes, biometric images, etc [1], [2], [3].

A digital image may be defined as a numerical representation of any scene. Thus, manipulating such images has become an easy task even for non-specialists, due to simplified tools available on any device, e.g., as smartphones and tablets. In this context, elements are combined to create a unique image with the potential to convince even the most experienced set of eyes [4]. The manipulation of images through forgery can impact the perception a viewer has of the depicted scene. Therefore, identifying unknown sources’ image authenticity has become of paramount importance, in the absence of any prior digital watermarking or other authentications techniques [5], [6].

Although the detection of image manipulation may present some similarities with semantic object detection, they differ in the context of the objective, since image manipulation detection focuses more on tampering artifacts instead of image content, which demands more sophisticated approaches. Therefore, distinguishing tempered images poses an increasing challenge, which requires the development of new techniques, as well as the enhancement of the ones developed so far, for proper identification of the image tampering operation performed over the models [7], [8].

Therefore, this work provides an overview of different research findings, and propose solutions to improve image anomaly detection (IAD) in the context of images tampering, using machine learning and other intelligent techniques, such as evolutionary computation, neural networks, fuzzy logic, and Bayesian reasoning. Moreover, it also presents some highlights concerning image tampering, as well as the approaches proposed to deal with the issues mentioned above through machine learning methods.

The organization of this work is presented as follows. Section 2 introduces a collection of researches related to traditional machine learning-based algorithms employed for image tampering detection (ITD) and further discusses the relevant contributions associated with anomaly detection (AD) methods applied to the same context. Section 3 presents the most commonly employed datasets for the task, as well as the intelligent techniques adopted in the proposed experiments, while Section 4 discusses the opportunities and challenges faced by the works considered in this survey. Finally, Section 5 states conclusions and a discussion about the future possibilities for research in image tampering.

In the last years, the interest of researchers towards image tampering detection and image anomaly detection increased considerably, due to their impact potential over different human actions. In this context, many works focused on developing feasible solutions to detected tampering in several image-related approaches, e.g., handwriting forgery [9], face spoofing [10], and machine learning-based image classifiers attack [11]. Moreover, image manipulation tools have been coupled to mobile devices, thus becoming popular on people daily’s life due to their simplicity, and also contributing to the emergence of image tampering.

Therefore, improvements towards image security become crucial, specifically considering image tampering detection [12], [13]. The field provides a brand new ground for research, whose demand grows to the same extent as manipulating tools become easier to handle and more popular, implying in the increasing number of fake images. Furthermore, in general, these fake images carry a high level of confidence, implying severe consequences to people and, at a higher level, even to governments and institutions [14].

However, as far as we are concerned, no work has presented an in-depth view regarding machine learning and intelligent system applications in the context of image tampering detection, which ends up being the main contribution of this survey.

This survey aims at providing an overview of the research progress related to security-related issues in image tampering detection. The scope of this work discusses methods of image tampering detection based on machine learning and evolutionary computation, among other intelligent systems.

The idea of this review is to provide information available on the current literature, as well as to be a new source for researchers interested in image detection and security issues. Additionally, it presents a clear vision of the possible challenges of current research and the new research guidelines.

Section snippets

Related works

The works presented in this survey highlights specific methods for image anomaly detection, focusing on a variety of autonomous intelligent systems. Besides, it also presents a sort of algorithms’ variations, which were improved for better performance in the context. Fig. 1 depicts a general pipeline applied for image anomaly detection employing intelligent techniques. Notice this architecture considers datasets collected and properly prepared for the proposed tasks, i.e., the data is processed

Methods and datasets

This section discusses the methodologies and datasets used in recent work focusing on Anomaly Detection for Hyperspectral Imagery.

In this context, the PCA skeleton kernel algorithm proposed by Olson et al. [85] performed very well over several methods and datasets. The method has proved to be reasonably robust to the variability of subsamples selection. Further, experiments were conducted over “The Hyperspectral Digital Imaging Collection Experiment (HYDICE)”, a dataset composed of 210

Discussion and open issues

The number of easy-to-use image editors has swiftly increased in the last decade, contributing to image manipulation even by lay users. However, such a phenomenon opens space for several risks concerning image security, such as real images tampering. Therefore, robust approaches and enhanced methods are desirable to avoid such incidents and provide more stable detection. Further, more studies are required to reduce the risk of potential threats in confidentiality and privacy, for instance,

Conclusions

This work highlighted the most significant works developed in the last years in the context of seam carved, tampered, and anomalous imagery detection through machine learning techniques. Besides, it also presents some advances regarding new tampering techniques, which poses new challenges to the field and demands a parallel advance in the detection techniques.

The works surveyed in this paper discusses, mainly, the concerns and efforts made by the scientific community and the industry towards

CRediT authorship contribution statement

Kelton A.P. da Costa: Conceptualization, Methodology, Software, Writing - original draft. João P. Papa: Conceptualization, Writing - review & editing. Leandro A. Passos: Conceptualization, Writing - review & editing. Danilo Colombo: Conceptualization, Writing - review & editing. Javier Del Ser: Validation, Writing - review & editing. Khan Muhammad: Formal analysis, Investigation, Validation, Writing - review & editing, Supervision. Victor Hugo C. de Albuquerque: Formal analysis, Investigation,

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

The authors are grateful to Fundação de Amparo á Pesquisa do Estado de São Paulo (FAPESP), Brazil grants #2017/22905-6, #2013/07375-0, #2014/12236-1, and #2016/19403-6 and by the Brazilian National Council for Research and Development (CNPq) via grants 4290032018-8, 3043152017-6, 4302742018-1,3070662017-7 and 4279682018-6, Petrobras, Brazil (grant201400545-0) for their financial support and Basque Government, Spain for its funding support through the ELKARTEK and EMAITEK funding

References (130)

  • VafadarM. et al.

    Hyperspectral anomaly detection using combined similarity criteria

    IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.

    (2018)
  • M. Ning, P. Yu, W. Shaojun, G. Wei, A weight SAE based hyperspectral image anomaly targets detection, in: 13th IEEE...
  • RajalakshmiC. et al.

    Study of image tampering and review of tampering detection techniques

    Int. J. Adv. Res. Comput. Sci.

    (2017)
  • L.F.S. Cieslak, K.A.P. Costa, J.P. Papa, Seam carving detection using convolutional neural networks, in: IEEE 12th...
  • ChangS. et al.

    BASO: A background-anomaly component projection and separation optimized filter for anomaly detection in hyperspectral images

    IEEE Trans. Geosci. Remote Sens.

    (2018)
  • M. Haselmann, D.P. Gruber, P. Tabatabai, Anomaly detection using deep learning based image completion , in: 17th IEEE...
  • A. Davy, T. Ehret, J. Morel, M. Delbracio, Reducing anomaly detection in images to detection in noise, in: 25th IEEE...
  • A. Megahed, S.M. Fadl, Q. Han, Q. Li, Handwriting forgery detection based on ink colour features, in: 8th IEEE...
  • ArashlooS.R. et al.

    An anomaly detection approach to face spoofing detection: A new formulation and evaluation protocol

    IEEE Access

    (2017)
  • D.J. Miller, Y. Wang, G. Kesidis, Anomaly detection of attacks (ADA) on DNN classifiers at test time, in: IEEE 28th...
  • ShankarK. et al.

    Adaptive optimal multi key based encryption for digital image security

    Concurr. Comput.: Pract. Exper.

    (2020)
  • PirbhulalS. et al.

    Mobility enabled security for optimizing IoT based intelligent applications

    IEEE Netw.

    (2020)
  • ArashdeepK. et al.

    High embedding capacity and robust audio watermarking for secure transmission using tamper detection

    ETRI J.

    (2018)
  • KitchenhamB. et al.

    Guidelines for Performing Systematic Literature Reviews in Software EngineeringTech. Rep. EBSE 2007–001

    (2007)
  • XuY. et al.

    Anomaly detection in hyperspectral images based on low-rank and sparse representation

    IEEE Trans. Geosci. Remote Sens.

    (2016)
  • D.K. Hoai, N. Van Phuong, Anomaly color detection on UAV images for search and rescue works, in: 9th International...
  • SunJ. et al.

    Learning sparse representation with variational auto-encoder for anomaly detection

    IEEE Access

    (2018)
  • K. Zhao, B. Liu, W. Li, N. Yu, Z. Liu, Anomaly detection and localization: A novel two-phase framework based on...
  • Q. Bammey, R. Grompone von Gioi, J. Morel, Automatic detection of demosaicing image artifacts and its use in tampering...
  • J. Schneible, A. Lu, Anomaly detection on the edge, in: IEEE Military Communications Conference, MILCOM, 2017, pp....
  • D.P. Sudharshan, S.A.H. Ameen, K. Baig, Dynamic detection of anomalies in pharmaceutical blisters using image...
  • Y. Pei, J. Weidong, T. Peng, Anomaly detection of railway catenary based on deep convolutional generative adversarial...
  • Molina-GarciaJ. et al.

    An effective fragile watermarking scheme for color image tampering detection and self-recovery

    Signal Process., Image Commun.

    (2019)
  • BaraniM.J. et al.

    A new digital image tamper detection algorithm based on integer wavelet transform and secured by encrypted authentication sequence with 3D quantum map

    Optik

    (2019)
  • HaghighiB.B. et al.

    TRLG: Fragile blind quad watermarking for image tamper detection and recovery by providing compact digests with optimized quality using LWT and GA

    Inform. Sci.

    (2019)
  • J. Dai, C. Deng, W. Wang, X. Liu, Low-rank and sparse tensor recovery for hyperspectral anomaly detection, in: IEEE...
  • T. Cheng, B. Wang, Manifold regularized low-rank representation for hyperspectral anomaly detection, in: IEEE...
  • ZhangL. et al.

    A tensor-based adaptive subspace detector for hyperspectral anomaly detection

    Int. J. Remote Sens.

    (2018)
  • YiZ. et al.

    A distributed parallel algorithm based on low-rank and sparse representation for anomaly detection in hyperspectral images

    Sensors

    (2018)
  • X. Ma, X. Zhang, N. Huyan, X. Tang, B. Hou, L. Jiao, Hyper-Laplacian regularized low-rank tensor decomposition for...
  • N. Patel, H. Soni, Anomaly detection using VCA algorithm for multi-temporal hyperspectral images, in: International...
  • H. Ju, Z. Liu, Y. Wang, Hyperspetral anomaly detection incorporating spatial information, in: Eighth International...
  • F. Küçük, B.U. Töreyin, F.V. Çelebi, Anomaly detection in hyperspectral data with matrix decomposition, in: 26th Signal...
  • ZhangL. et al.

    A spectral-spatial method based on low-rank and sparse matrix decomposition for hyperspectral anomaly detection

    Int. J. Remote Sens.

    (2017)
  • S. Shashikar, V. Upadhyaya, Traffic surveillance and anomaly detection using image processing, in: Fourth International...
  • N. Patil, P.K. Biswas, Video anomaly detection and localization using 3D SL-HOF descriptor, in: Ninth International...
  • S. Zaidi, B. Jagadeesh, K.V. Sudheesh, A.A. Audre, Video anomaly detection and classification for human activity...
  • K. Takuya, F. Syoji, Y. Hiroki, N. Masashi, I. Yoshio, Anomaly detection using local regions in road images acquired...
  • J.C. SanMiguel, J.M. Martínez, L. Caro-Campos, Object-size invariant anomaly detection in video-surveillance, in:...
  • Y. Wang, B. Xue, L. Wang, H. Li, L. Lee, C. Yu, M. Song, S. Li, C. Chang, Iterative anomaly detection, in: IEEE...
  • Cited by (0)

    View full text