Abstract
Detecting manipulations in images is becoming increasingly important for combating misinformation and forgery. While recent advances in computer vision have lead to improved methods for detecting spliced images, most state-of-the-art methods fail when applied to images containing mostly text, such as images of documents. We propose a deep-learning method for detecting manipulations in images of documents which leverages the unique structured nature of these images in comparison with those of natural scenes. Specifically, we re-frame the classic image splice detection problem as a node classification problem, in which Optical Character Recognition (OCR) bounding boxes form nodes and edges are added according to a text-specific distance heuristic. We propose a Variational Autoencoder (VAE)-based embedding algorithm, which when combined with a graph neural network with attention, outperforms both a state-of-the-art image splice detection method and a document-specific method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Change history
28 June 2023
A correction has been published.
Notes
- 1.
The images in this dataset are images from PDFs from academic works. The PDFs include articles from sociology journals, some of which discuss violent content and may be upsetting to certain readers.
References
Ahmed, A.G.H., Shafait, F.: Forgery detection based on intrinsic document contents. In: 2014 11th IAPR International Workshop on Document Analysis Systems, pp. 252–256. IEEE (2014)
Bayar, B., Stamm, M.C.: A deep learning approach to universal image manipulation detection using a new convolutional layer. In: ACM Workshop on Information Hiding and Multimedia Security, pp. 5–10 (2016)
Bertrand, R., Gomez-Kramer, P., Terrades, O.R., Franco, P., Ogier, J.M.: A system based on intrinsic features for fraudulent document detection. In: ICDAR, pp. 106–110. IEEE (2013)
Bertrand, R., Terrades, O.R., Gomez-Kramer, P., Franco, P., Ogier, J.M.: A conditional random field model for font forgery detection. In: ICDAR, pp. 576–580. IEEE (2015)
van Beusekom, J., Shafait, F., Breuel, T.M.: Automatic authentication of color laser print-outs using machine identification codes. Pattern Anal. Appl. 16(4), 663–678 (2012). https://doi.org/10.1007/s10044-012-0287-5
van Beusekom, J., Shafait, F., Breuel, T.M.: Text-line examination for document forgery detection. IJDAR 16(2), 189–207 (2013)
Bronstein, M.M., Bruna, J., LeCun, Y., Szlam, A., Vandergheynst, P.: Geometric deep learning: going beyond Euclidean data. IEEE Signal Process. Mag. 34(4), 18–42 (2017)
Chernyshova, Y.S., Aliev, M.A., Gushchanskaia, E.S., Sheshkus, A.V.: Optical font recognition in smartphone-captured images and its applicability for ID forgery detection. In: ICMV 2018, vol. 11041, p. 110411J (2019)
Cozzolino, D., Verdoliva, L.: Noiseprint: a CNN-based camera model fingerprint. IEEE Trans. Inf. Forensics Secur. 15, 144–159 (2019)
Cruz, F., Sidere, N., Coustaty, M., D’Andecy, V.P., Ogier, J.M.: Local binary patterns for document forgery detection. In: ICDAR, vol. 1, pp. 1223–1228. IEEE (2017)
Ghosh, A., Zhong, Z., Boult, T.E., Singh, M.: SpliceRadar: a learned method for blind image forensics. In: CVPR Workshops, pp. 72–79 (2019)
Gupta, A., Saxena, N., Vasistha, S.: Detecting copy move forgery using DCT. Int. J. Sci. Res. Publ. 3(5), 1 (2013)
Gupta, S., Kumar, M.: Forensic document examination system using boosting and bagging methodologies. Soft. Comput. 24(7), 5409–5426 (2019). https://doi.org/10.1007/s00500-019-04297-5
Hassan, T.: ICDAR 2013 table competition dataset (2013)
Hu, M.K.: Visual pattern recognition by moment invariants. IRE Trans. Inf. Theory 8(2), 179–187 (1962)
Hu, W.C., Chen, W.H.: Effective forgery detection using DCT+ SVD-based watermarking for region of interest in key frames of vision-based surveillance. Int. J. Comput. Sci. Eng. 8(4), 297–305 (2013)
Hu, W.-C., Chen, W.-H., Huang, D.-Y., Yang, C.-Y.: Effective image forgery detection of tampered foreground or background image based on image watermarking and alpha mattes. Multimedia Tools Appl. 75(6), 3495–3516 (2015). https://doi.org/10.1007/s11042-015-2449-0
Huh, M., Liu, A., Owens, A., Efros, A.A.: Fighting fake news: image splice detection via learned self-consistency. In: ECCV, pp. 101–117 (2018)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. In: 2nd International Conference on Learning Representations, ICLR 2014, Conference Track Proceedings, Banff, AB, Canada, 14–16 April 2014 (2014)
Kniaz, V.V., Knyaz, V., Remondino, F.: The point where reality meets fantasy: mixed adversarial generators for image splice detection (2019)
Korus, P., Huang, J.: Evaluation of random field models in multi-modal unsupervised tampering localization. In: 2016 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–6 (2016)
Liu, B., Pun, C.M.: Splicing forgery exposure in digital image by detecting noise discrepancies. Int. J. Comput. Commun. Eng. 4(1), 33 (2015)
Mayer, O., Stamm, M.C.: Exposing fake images with forensic similarity graphs. IEEE J. Sel. Top. Signal Process. 14(5), 1049–1064 (2020)
McInnes, L., Healy, J., Melville, J.: UMAP: uniform manifold approximation and projection for dimension reduction (2018). http://arxiv.org/abs/1802.03426. Comment: Reference implementation available at http://github.com/lmcinnes/umap
Ng, T.T., Chang, S.F., Sun, Q.: A data set of authentic and spliced image blocks. ADVENT Technical Report, Columbia University, pp. 203–2004 (2004)
Pun, C.M., Liu, B., Yuan, X.C.: Multi-scale noise estimation for image splicing forgery detection. J. Vis. Commun. Image Represent. 38, 195–206 (2016)
Roy, A., Dixit, R., Naskar, R., Chakraborty, R.S.: Copy-move forgery detection with similar but genuine objects. In: Digital Image Forensics. SCI, vol. 755, pp. 65–77. Springer, Singapore (2020). https://doi.org/10.1007/978-981-10-7644-2_5
Shang, S., Kong, X., You, X.: Document forgery detection using distortion mutation of geometric parameters in characters. J. Electron. Imaging 24, 023008 (2015)
Shang, S., Memon, N., Kong, X.: Detecting documents forged by printing and copying. EURASIP J. Adv. Signal Process. 2014(1), 1–13 (2014). https://doi.org/10.1186/1687-6180-2014-140
Smith, R.: An overview of the Tesseract OCR engine. In: ICDAR 2007, vol. 2, pp. 629–633. IEEE (2007)
van Beusekom, J., Stahl, A., Shafait, F.: Lessons learned from automatic forgery detection in over 100,000 invoices. In: Garain, U., Shafait, F. (eds.) IWCF 2012/2014. LNCS, vol. 8915, pp. 130–142. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20125-2_12
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Wen, L., Qi, H., Lyu, S.: Contrast enhancement estimation for digital image forensics. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 14(2), 1–21 (2018)
Wu, Y., AbdAlmageed, W., Natarajan, P.: ManTra-Net: Manipulation tracing network for detection and localization of image forgeries with anomalous features. In: CVPR, pp. 9543–9552 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Joren, H., Gupta, O., Raviv, D. (2022). Learning Document Graphs with Attention for Image Manipulation Detection. In: El Yacoubi, M., Granger, E., Yuen, P.C., Pal, U., Vincent, N. (eds) Pattern Recognition and Artificial Intelligence. ICPRAI 2022. Lecture Notes in Computer Science, vol 13363. Springer, Cham. https://doi.org/10.1007/978-3-031-09037-0_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-09037-0_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09036-3
Online ISBN: 978-3-031-09037-0
eBook Packages: Computer ScienceComputer Science (R0)