Document manipulation is a recently arising problem, especially with the rapid spread of fabrication technology. The tools to alter documents are now publicly available and can result in high quality forgeries, indistinguishable from genuine ones. Forged documents may wreak havoc on many processes dependent on the validity of the document, leading to lasting consequences such as financial loss. Therefore, the process of identifying a document that has been altered is essential. A system that is capable of scrutinizing documents as either forged or genuine through discriminative features (such as distortions or character misalignment) can assist industries with heavily reliance on documents for processes such as identity verification. Most of the documents involved in such processes have sufficiently complex backgrounds. We present a computer-vision-based system that detects changes in the background of the aforementioned documents as a result of manipulations made to its contents through the use of image subtraction. The system takes an image as input and then classifies the document as genuine or forged. Our proposed system produces an accuracy of 95% using CNN on unaligned images as well as 100% for aligned images.