There is a vast store of scientific knowledge contained in traditional archival media, such as paper, film, photographs, etc., created prior to the development of portable formats for documents. There are very useful scientific summaries such as tables and graphs embedded within these documents. While converting these to documents to images is straightforward, identifying these artifacts automatically is still challenging. Researchers have shown interest in this area by proposing numerous techniques for the detection of such artifacts. In this paper we review previous work, and present a comparison of three deep learning algorithms that address this problem. A dataset comprising of the ICDAR 2013 benchmark data set supplemented with data from the “wild” is used to train and test the models. The models are compared on ease of training, and accuracy of each model. The results of this comparison and practical suggestions for using deep learning models for document image recognition are presented.