1 Introduction

JPEG (Joint Photographic Experts Group) image format is considered the most popular image format for Internet images and personal photographs. We rarely find any consumer cameras and image editing softwares that come without JPEG image encoder and decoder. For a camera image of JPEG format, in-camera compression refers to the primary compression it undergoes in the camera and out-camera compression is a recompression performed out of the camera. Out-camera recompression happens when the camera image is sent as an email attachment, resized, cropped or recomposed [11] for image forgery, image color adjustment or contrast enhancement [13]. Recompression is also a common step for adding visibly imperceptible signal onto an image which is done for digital watermarking and steganography [4, 14]. Knowing the compression history of the image offers digital forensics investigators important clues or telltale signs for detecting image splicing [23] or other image manipulations. The knowledge of compression history can also make steganalysis more effective [7]. This paper pays particular attention to identifying in-camera compression from out-camera compression. It is extremely important to manifest the primary compression of an image in situations such as for photo authentication in journalism or in a photography competition. For example, we can identify a photorealistic computer generated image if it can not present the in-camera compression although its owner claims that it is a camera image and just has been recompressed by image editing software. Similarly, the owner of a doctored camera image may claim that it just has been resaved out of the camera. The missing of the in-camera compression can be regarded as evidence that it is nothing but a forgery. In such cases, an illicit photography work can be exposed. In summary, in-camera compression detection may be helpful for identifying image authenticity and for establishing rules of image manipulation in journalism or in a photography competition.

Kornblum [16] uses the quantization table from JPEG headers to identify camera images from those processed by software. Kee et al. [15] attempt to figure out whether an image has been compressed by the camera or by Adobe Photoshop through extracting a camera signature from the metadata in the JPEG file header. The camera signature consists of information about quantization tables, Huffman codes, thumbnails, and EXIF format. However, given a camera image, for example one from the Internet, there is no easy way to find out how many times this image have been compressed and at what quality factor. The metadata of the image reveals nothing about such compression history beyond the current compression.

Double JPEG compression detection evaluates the characteristics of the quantization coefficients of a JPEG image and decides whether an image has been JPEG compressed once or twice [3, 8, 17, 19, 20, 25]. The telltale signs for double JPEG compression are the zeros and double peaks that lie on the distribution of the JPEG coefficients of a DCT mode [20]. Fu et al. [8] and Li et al. [17] showed that the distributions of the first digit of the JPEG coefficients can be used to detect double JPEG compressed images. Chen et al. [3] considered the correlations between neighboring JPEG coefficients as features to detect double JPEG compression. Milani et al. [22] extended the work by Li et al. [17] and used the first-digit distribution information to infer the JPEG compression count up to four prior compression. Huang et al. [12] addressed the same problem of compression count estimation for the special case where the successive compressions are using the same quantization table.

Estimating the JPEG quantization steps is important for specific cases such as steganalysis [7] and digital image forensics. The spatial integer round-off effect may introduce minor correlation between DCT coefficients of different frequency modes. Despite this, the quantization effect on the DCT coefficients of each frequency mode can be considered approximately independent assuming no JPEG block alignment between successive rounds of JPEG compression. Therefore, the estimation of the JPEG quantization step can be treated independently for each respective DCT mode. In general, when block DCT transform is performed on an image which has been previously JPEG compressed, the distribution for the DCT coefficients for a frequency mode is periodic. The periodicity of the DCT coefficient distribution corresponds to the quantization step size of the previous JPEG compression. This property has been used by many previous works to estimate the JPEG quantization step of a bitmap image that has been JPEG compressed [5, 8, 9, 18, 21, 28]. In the case of double JPEG compression, some methods have been presented for estimating the quantization table for the first JPEG compression [20, 24]. Lukáš and Fridrich [20] estimated the quantization steps for the low-frequency DCT coefficients of AC mode by matching the JPEG coefficient distributions to the simulated ones or by learning the characteristics of these distributions. Pevnỳ and Fridrich [24] went further by finding the closest standard quantization matrix once the quantization steps for a small set of DCT coefficients of AC mode are estimated.

However, estimating the quantization table of the primary JPEG compression for a doubly compressed image is often difficult. It is because many DCT coefficients of high frequency are quantized to zero at low quality factors [9, 20]. Moreover, a large number of images are needed if Machine learning is applied to the quantization table estimation. To the best of our knowledge, no methods that can estimate the quality factor of the primary compression along with identifying the in-camera compression are reported. In this paper we particularly address the problem of detecting the in-camera JPEG compression of an image. Our work demonstrates the possibility of detecting the in-camera compression along with estimating the quality factors of historical compressions. This is done through a recompression operator [6]. We measure the error between the given image and the recompressed one in the YCbCr color space. Unlike the methods in the prior works, both the quantization and the chroma subsampling effect are considered in our investigation. The chroma subsampling effect offers telltale signs for whether an image is compressed within or outside a camera.

This rest of the paper is organized as follows. First, we provide a brief introduction to the JPEG compression procedure. Additionally, variations of subsampling type and quantization table are discussed. Then, we present our method of detecting in-camera JPEG compression based on the analysis of JPEG error. After that, the experimental results and discussions are presented. Finally, in the last section we conclude our paper.

2 JPEG compression and variations

The lossy JPEG compression scheme is the most widely used technique for photographic images. In this section, we first provide a brief introduction to JPEG compression. Then, especially, we discuss the preference of different JPEG compression implements about the chroma subsampling and the quantization tables.

2.1 JPEG compression

Given a color image in RGB color space, the standard lossy JPEG compression procedure consists of the following steps:

Color space transformation

The standard JPEG compression only supports YCbCr (also known as YUV) color model, where Y is the luminance channel while Cb and Cr are the chrominance channels. YCbCr color space is conducive for image compression as an image is conveniently separated into the Y channel, which is dominant in preserving perceptual quality, and the chrominance channels. Thus, a color image needs to be converted into YCbCr color space before JPEG compression.

Chroma subsampling

Chroma subsampling is used to reduce the resolution of the chrominance channels. Humans are less sensitive to the detailed spatial variation in the chrominance channels as compared to that in the luminance channel as human eyes have sparser color-sensitive receptors. Therefore, chrominance channels can be more aggressively subsampled without significant impact on the perceptual quality of an image. For example, if a 2 × 2 subsampling is imposed in the chrominance components, the chrominance images are broken into 2 × 2 pixels blocks and only the average value of each block is kept. As a result, the final resolution of the chrominance images is reduced by half in both the horizontal and vertical directions. Figure 1 shows a common JPEG subsampling scheme with 1 × 1 subsampling (i.e., no subsampling) for the luminance channels and 2 × 2 for the chrominance channels.

Fig. 1
figure 1

Relative sizes corresponding to different color channels with 2 × 2 chroma subsampling

Full size image

Block DCT transform

The image is divided into blocks of 8 × 8 pixels. A discrete cosine transform (DCT) is applied to all non-overlapping 8 × 8 blocks in each channel:

$$ d_{ij} = \sum\limits^{7}_{x,y=0} \frac{1}{4}c(i)c(j)g_{xy}\cos\left(\frac{i\pi}{16}(2x+1)\right)cos\left(\frac{j\pi}{16}(2y+1)\right), $$
(1)

where \(c(0)=1/{\sqrt {2}}\) and c(ω > 0) = 1, g xy is the pixel value at coordinates (x, y), and d ij is the DCT coefficients at coordinates (i, j) .

Quantization

Each DCT coefficient d ij in an 8 × 8 block is quantized by the corresponding quantization step Q ij as specified in the quantization matrix Q:

$$ D_{ij} = \left[ \frac{d_{ij}}{Q_{ij}} \right], \forall i,j \in \{0,\ldots,7\}, $$
(2)

where [·] is an integer rounding operator, i.e., [·] ::=truncate(round(·)). The quantized integers D ij are called JPEG coefficients. This quantization step makes JPEG compression lossy. The greater the quantization step size, the greater the loss in image quality and conversely the more efficient the compression. In practice, the quality of an JPEG image is controlled by a discrete set of JPEG quality factors corresponding to a quantization matrix which defines the quantization step sizes. It has been shown that quantization matrix that results in relatively greater error variance in the lower DCT frequency modes can lead to blocky artifacts in the spatial domain [26].

Entropy encoding

Entropy encoding is a form of lossless source coding where the JPEG coefficients are scanned in zigzag order before runlength encoding and Huffman encoding (or arithmetic encoding) are applied.

2.2 Variations

There are various parameters that can be customized in JPEG compression. Different JPEG encoders can have different set of quantization matrices, different Huffman coding table or subsampling type [15]. In this paper we characterize a JPEG compression with its compression implement, quality factor and chroma subsampling type.

JPEG compression quality is defined by the quantization tables (each with an array of 64 values). In theory each color image has three quantization tables respectively for Y, Cb and Cr channels. However, the same quantization table is often employed in Cb and Cr channels [15]. Shown in Table 1 are a pair of quantization tables published by the Independent JPEG Group(IJG) [16]. They corresponds to a quality factor of 50. A scaling factor are used to generate the quantization tables of a quality factor from 1 to 100 by scaling the coefficients in the base tables. These quantization tables are customarily called as standard quantization tables. Many commercial digital cameras and programs use the standard quantization tables. Some programs such as Adobe Photoshop employ non-IJG quantization tables [16]. Some brands of camera sometimes may use custom quantization tables depending on the processed image. For example, we found some test images from two SAMSUNG cameras have modified quantization tables different from the standard quantization tables. Similar report about a Fuji camera can be seen in [16].

Table 1 Standard JPEG quantization tables (QF=50)
Full size table

Most digital camera manufacturers prefer 2 × 1 subsampling (i.e., 2 to 1 subsampling in the column direction) in chrominance channels, while most popularly used JPEG compression programs prefer 2 × 2 subsampling [2]. Adobe Photoshop implements its own proprietary JPEG compression algorithm [27] and defines its compression quality ranging from level 0 to 12 instead of the common scale of 0-100 %. In Adobe Photoshop, 1 × 1 subsampling (i.e., no subsampling) is imposed on the chrominance channels for level 7 to 12, while 2 × 2 subsampling for levels 0 to 6.

3 Our method of identifying in-camera JPEG compression

In this section, we describe our experimental analysis based on JPEG error and present our method of identifying in-camera compression from out-camera compression. For convenience, we denote a single round of JPEG compression as QF q(T), where q is the quality factor number and T denotes the compression tool. For example, QF85(M) indicates a compression with the MATLAB JPEG compression function at a quality factor of 85. If an image is firstly compressed inside a camera at a quality factor of 85 followed by a compression with MATLAB tool at 90, we use QF85(C)-QF90(M) to represent its compression history. The chroma subsampling scheme is not denoted explicitly as it is in general closely related to a compression tool.

3.1 Probing in-camera JPEG compression

When an image is recompressed with the same quantization tables as that of the previous compression, the change in the image intensity is insignificant. Assuming that the image has been JPEG compressed twice and the quantization steps of the first compression is significantly larger than those of the second, i.e., the first quality factor is less than the second, recompressing the doubly compressed image with the same quantization tables as that of the first compression will also result in insignificant changes in the image intensity although the changes are more severe than the former case. Farid [6] considered the changes in the latter case as JPEG ghost. Farid directly computed the sum total of square difference from pixel values of R, G and B color components by employing recompression operator. In this paper, we compute the sum of square difference of pixel values in Y, Cb and Cr color channels, respectively. There are two major reasons to select YCbCr color space. First, YCbCr color space is used in JPEG compression. We deem that YCbCr color space is a priority to analyze the error introduced by the JPEG compression. Second, as described in Section 2, different types of chroma subsampling are applied in the chrominance channels by different JPEG compression tools. Especially, most digital camera manufacturers prefer 2 × 1 subsampling while most JPEG compression tools prefer 2 × 2 subsampling in chrominance channels. Subsampling is rarely applied in luminance channel. Thus, we can investigate the chroma subsampling effect in JPEG compression. We expect it can help us with identifying in-camera JPEG compression from out-camera JPEG compression by comparing the d q curves of luminance and chrominance components. Such effect is often overlooked by previous methods.

For a given image without knowing its JPEG compression history, we recompress it with quality factors ranging from q 1 to q 2 in increments of 1, i.e., Q T = [q 1: 1: q 2]. The sum of square difference of pixel values in a certain color channel is computed:

$$\begin{array}{@{}rcl@{}} \delta_{q}^c=\sum\limits_{x,y}\varepsilon^{c}(x,y,q)=\sum\limits_{x,y}\left[{\vphantom{g^{c}}}\right.(g^{c}(x,y)-g_{q}^{c}(x,y)\left.{\vphantom{g^{c}}}\right]^{2}, \end{array} $$
(3)

where g c(x,y) is the pixel value at coordinates (x,y), \(g_{q}^{c}(x,y)\) is the result of compression g c(x,y) at a quality factor q, q∈Q T , and c∈{Y,Cb,Cr}. Then, \(\delta _{q}^{c}\) of each quality factor q is scaled into the range [0,1],

$$ d_{q}^c=\frac{\delta_{q}^c-\min(\delta_{q}^c)}{\max(\delta_{q}^c)-\min(\delta_{q}^c)}. $$
(4)

Finally, we draw a curve of d q over the range of Q T for each color component.

In this paper, we particularly take into consideration different tools with the same quantization table definition, e.g., the standard quantization tables, but with different subsampling type. Hence MATLAB is used as a recompression encoder for the in-camera compression detection because it employs the standard quantization tables as many consumer-end cameras do but has a difference in chroma subsampling type from cameras. We observe the effects of quantization and subsampling in the luminance and the chrominance channels at the same time.

Figure 2 shows a natural-scene image taken by a Nikon E7900 camera and saved in JPEG format. 2 × 1 subsampling is performed in its two chrominance channels while no subsampling in the luminance channel. The original resolution of the image is 2592 × 1944 pixels. JPEGsnoop [2] is employed to obtain quantization tables, subsampling type as well as other detailed information of the image. We compress the camera image at a quality of 92 in MATLAB. Then the original image and its compressed version is recompressed by a range of quality factors from 60 to 100. Figs. 3a and b respectively illustrate their d q curves in three color channels. To see the results more clearly, we use different line types to indicate the compression of different JPEG tools. For example, the camera’s compression is indicated by dotted line and MATLAB’s by dashed line.

Fig. 2
figure 2

The test image for the analysis experiment on compression history

Full size image
Fig. 3
figure 3

Applying recompression operator on a camera image (shown in Fig. 2) and its doubly compressed version. a d q curves in Y, Cb and Cr color channels of the singly JPEG compressed camera image. b d q curves in Y, Cb and Cr color channels of the doubly JPEG compressed camera image

Full size image

We note that in Fig. 3a the luminance d q curve in the mass decreases with the increase of QF (quality factor) until a local minimum is reached at QF 79. By contrast, the two chrominance curves have not such an obvious change trend, especially the local minimum at QF 79. In Fig. 3b, however, we observe that both the chrominance d q curves of the doubly compressed image continuously decline with the increase of QF until a minimum is reached at QF 92. At the same time, we observe that the luminance d q curve in Fig. 3b displays two local minima at QF 79 and 92, respectively. That is, the three d q curves only coincide at QF 92 which corresponds to the second quality factor. The luminance d q curve still singlehandedly presents the conspicuous local minimum at QF 79, which corresponds to the primary compression in camera. Besides, we observe that the d q value of QF 79 is larger than that of QF 92 in Fig. 3b. It suggests that the earlier a compression is implemented, the larger the corresponding local minimum is, and the most recent compression has the global minimum on d q curves without considering those larger QFs. This conclusion can be applied to the generalized case of multi-compression. As will be seen in Section 3.3, it will be used in our proposed method to remove those false minima.

Farid [6] has shown that the higher quality factor prior to the most recent compression cannot be reliably detected with a recompression operator. Hence, we can pay no attention to a local minimum if its corresponding QF is larger than that of the most recent compression. For example, we can ignore the local minima at QF 89 in Fig. 3a. It does not indicate a true historical JPEG compression.

The recompression operator using MATLAB tool is applied to the camera image in the above experiments. In fact, this implies a transition of JPEG compression tools. That is, the primary compression is implemented by camera JPEG encoder while the recompression by MATLAB. In this case, different chroma subsampling types are applied to the primary compression in the camera (chroma subsampling type: 2 × 1) and the recompressions in MATLAB (chroma subsampling type: 2 × 2). Because of the transition of subsampling type, the chrominance d q curves cannot present the local minimum of the primary compression. Especially, subsampling effect is larger for the original camera image than its doubly compressed version. This can be demonstrated by the abnormal behavior of the chrominance d q curves in Fig. 3a. On the other hand, because subsampling is not performed, we can find the corresponding minimum of the primary compression in the luminance channel. For the doubly compressed image, both the second compression and the recompression (i.e., the third compression) are implemented in MATLAB(no transition of subsampling type), this accounts for the coinstantaneous presence of the local minimum at QF 92 on three d q curves.

3.2 Effect of image content on d q curves

Note that good performance of d q curves are dependent on the image content. To demonstrate it, we crop an image into blocks of different complexity. The test image has undergone in-camera compression at a quality of 80 and is doubly compressed at 90 in MATLAB. If we chance to select an image block of largely uniform intensity (e.g., sky in background) rather than a textured region, recompression may result in a relatively very small error [6] on this image block. In this case, the d q curves of this image block are vulnerable to noise although the error has been normalized in term of Eq. (4). It is generally known that the larger variance of an image block is, the more dramatically the image intensity changes. Hence, we use variance to measure the block complexity. Shown in Figs. 4a and b are the different series of d q curves of a camera image when we crop it with a size of 256 × 256 in the upper left corner and near the center, respectively. The two image blocks have rather different content. The former has block variance 3.08 while the latter 7.11E3.

Fig. 4
figure 4

Different appearances of d q curves when cropping image blocks with different image content from a doubly compressed image. The 256 × 256red boxes in the left two panels denote the cropped image block for the detection. a A smoothing block. The variance of the image block is 3.08. b A relatively complex block. The variance of the image block is 7.11E3. The original image is taken by a NIKON E7900 camera. It is compressed at a quality 80 in the camera and undergoes a second compression at a quality 90

Full size image

First, we observe the un-smooth d q curves shown in the right panel of Fig. 4a. Those d q curves correspond to image block in the upper left corner as shown in the left panel of Fig. 4a (highlighted by the red box). Besides the minimum at QF 90, a perceptible minimum occurs at QF 80 on the three d q curves at the same time. As a result, the in-camera compression may not be identified by the proposed algorithms. We assume that this is because the subsampling of chrominance channels has less impact on the uniformly black image block. Thus, the local minimum error in chrominance channels is also achieved when the image is recompressed with the same quantization tables as the original, in spite of different subsampling types between the two JPEG compressions (i.e., the recompression in MATLAB and the first compression in the camera). In contrast, the d q curves shown in the right panel of Fig. 4b are much smoother when the image block is cropped near the center (shown in the left panel of Fig. 4b and also highlighted by the red box). In this case, the in-camera compression consequently can be correctly identified due to a dramatic change of intensity in this image block. The analysis reminds us to consider the content of the cropped image block. As will be seen in Section 4, we try to obtain smooth d q curves by avoiding uniform regions with small amounts of high spatial frequency content, for example, the uniform sky when cropping an image for detecting the in-camera compression.

3.3 Our algorithms

As seen in Fig. 3, regardless of the source of the JPEG compressions imposed on the image, the luminance d q curve generally presents local minima more perfectly corresponding to the quality factors of all JPEG compressions than the chrominance d q curves do. In contrast, the chrominance d q curves cannot present the minima at the corresponding QF with the transition of subsampling type in the chrominance channels. Such transition may happen when a JPEG image from a camera is recompressed by a JPEG compression software or an image editing software. This can help us to identify in-camera compression from out-camera compression by observing the absence of the first minimum on the chrominance d q curves compared to the luminance d q curve.

MATLAB, which we use as the recompression encoder, has the same quantization table as that of the in-camera compression but different subsampling type. Hence we can identify the in-camera compression by drawing the d q curves through imposing recompression on the image using MATLAB. However, this may not be always true for other JPEG encoders. For example, as mentioned before, Adobe Photoshop defines its own JPEG compression parameters ranging from level 0 to 12 instead of the common scale of 0-100 %. If we recompress the camera image with Adobe Photoshop, the in-camera compression may not be exposed.

Although the in-camera compression can be easily identified by drawing d q curves, we still develop a system of automatically identifying the in-camera compression for a doubly compressed image. A block diagram of the whole system is shown in Fig. 5. The first step consists of generating d q curve of Y, Cb and Cr color channels. Local minima of each d q curve are detected in the second step and the false local minima are removed in the third step. So far, a primary JPEG compression sequence is obtained in Y, Cb, and Cr color channel, respectively. In the last step, the in-camera compression is identified by examining whether the first minimum on the luminance d q curve is present on the chrominance curves, and the final JPEG compression sequence is obtained. Our detection method are summarized in Algorithms 1 and 2.

Fig. 5
figure 5

Block diagram of the proposed framework

Full size image
figure c
figure d

Algorithm 1 describes the first three steps. The input image is recompressed with quality factors in the sequence of Q T . The error(i.e., d q ) between the input image and its recompressed versions is computed in Y, Cb and Cr, respectively. Then, local minimum detecting on three d q curves is performed. A local minimum may result from a historical compression. However, an undesirable minimum may occur if the quantization steps of a recompression happen to be divisors of those of the historical compression. In this case, false compression detection may be caused. We can remove those false minima in term of the rule that the earlier a compress is implemented, the larger the corresponding local minimum is. Finally, three compression sequences of Y, Cb and Cr channels are output. Each compression sequence includes the QFs from the earliest to the most recent compression. Note that here we suppose that the image may have undergone multiple compressions to make the proposed method more general, as follows.

Algorithm 2 describes the last step, in which in- and out-camera compression detection is performed. The earliest compression of Y channel is identified as in-camera compression if the corresponding QF does not exist in Cb and Cr compression sequence. Except the in-camera compression, an out-camera compression is one whose QF occurs in Y compression sequence and at least in one of Cb and Cr. Finally, Algorithm 2 outputs the quality factor of the in-camera and the final compression sequence.

4 Experiments and discussions

In this section, we evaluate the efficacy of the proposed detection method of in-camera JPEG compression. In addition, we present full discussions to the proposed method in the last subsection.

4.1 Datasets

We collected 1,000 camera images from 17 consumer-end cameras. The involved camera brands include Canon, Sony, Olympus, Samsung and Nikon. Those images contain diverse content with a wide range of shutter time, weather, and environment condition (outdoor/indoor, daylight/night/rainy), and various object types (architecture/person/ animal/ plant/natural scene). Figure 6 shows some representative samples from the dataset. The primary quantization tables of each camera image is gotten by JPEGsnoop [2]. At the same time, we obtain the corresponding quality factor scaled into {1, 2,...100}. The quality factors of those images vary from 68 to 96.

Fig. 6
figure 6

Samples from our JPEG camera image set, collected from 17 consumer-end cameras with different brands and models

Full size image

Our goal is to identify in-camera compression from out-camera compression for a doubly compressed image. Thus all camera images are compressed again at a quality of 98 or 100 in MATLAB. Note that a quality factor of 100 means the highest compression quality rather than a lossless compression or non-compression. In addition, 155 low quality images (QF from 68 to 85) chosen from the 1000 camera images are doubly compressed at a quality of 90. All the doubly compressed images are used as test images in our experiments.

In the meantime, 500 RAW-format (CR2) images are downloaded from BOSSBase [1]. Some sample images from BOSSBase are shown in Fig. 7. Those never-compressed images are used as test images after resaving them as TIF-format in Adobe Photoshop CS5. Then those TIF images are doubly compressed in MATLAB with different quality combinations.

Fig. 7
figure 7

Samples from BOSSBase image set

Full size image

4.2 Results

Our test experiments are carried out on image blocks of different sizes. All images are cropped with sizes ranging from 512 × 512 to 8 × 8. Image blocks are randomly selected from images and all have large variance to avoid smoothing blocks. Please note, image blocks of different sizes for each image are cropped at the same position with the upper-left alignment. This make our comparison of detection performance of different block sizes more reasonable. In any case, the original 8 × 8 JPEG grid of an image is preserved when cropping.

We hope that the in-camera compression of an image can be exposed with a probability as high as possible while the out-camera compression is misreported as in-camera with a probability as low as possible. Our proposed detection method, firstly, must detect the quality factor of the first compression from a doubly compressed image. That is, if the quality factor of the first compression from the image is not correctly detected, the detection is treated as an error although the source of the first compression, i.e., in-camera or out-camera, is identified. Tables 2 and 3 show the experimental results. It is shown that when we use a larger size of image block, e.g. 512 × 512, our method can achieve a satisfactory power of identifying in-camera compression from out-camera compression. But we also can get correct detection result with a small size of image block if it contains sufficient information in spatial frequency. In general, The proposed method works better if an image has undergone a higher quality compression. However, a large enough difference between the first and the second quality factor is needed for the detection.

Table 2 Accuracy rate (%) of in-camera JPEG compression detection for doubly compressed camera images from our JPEG camera image set with different sizes of detection block
Full size table
Table 3 Error rate (%) of out-camera JPEG compression detection for doubly compressed BOSSBase images with different sizes of detection block
Full size table

4.3 Discussions

Color space selection

In our proposed method, we compute the JPEG error in Y, Cb and Cr color channels and obtain good performance to identify the in-camera compression. But what will it be if we do that in R, G and B color channels? Figure 8a and b respectively illustrate the d q curves of R, G and B channels for the original image shown in Fig. 2 and its doubly compressed version. We observe that the local minimum occurs nearly simultaneously at QF=79 (the quality factor of the in-camera compression) in three channels (especially, in R and G channel) both in Fig. 8a and b. Compared to Fig. 3a and b, the in-camera compression is indistinguishable from the out-camera compression in Fig. 8a and b according to our identification criterions. That is, other identification criterions have to be considered if we use RGB color space. As we have described in Section 2, JPEG compression uses YCbCr color space and subsampling is applied only in chrominance components. YCbCr is converted to RGB as follows [10]

$$\begin{array}{@{}rcl@{}} \begin{array}{lll} R&=&Y+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~1.402~~(C_r-128)\\ G&=&Y-0.34414(C_b-128)-0.71414(C_r-128)\\ B&=&Y+1.772~~~(C_b-128)~~~~~~~~~~~~~~~~~~~~~~~~~~. \end{array} \end{array} $$
(5)

Obviously, luminance and chrominance components are mixed in RGB so that we cannot separate chroma subsampling effect in RGB color space as easily as in YCbCr. We deem that it is the reason of the different performance of d q curves in the two color spaces.

Fig. 8
figure 8

d q curves of R, G and B color channels for (a) the singly JPEG compressed camera image and (b) the doubly JPEG compressed camera image. The experimental image is shown in Fig. 2

Full size image

Non-IJG quantization tables

As mentioned before, some cameras do not completely conform to the IJG standard. For example, we found 8 out of 120 images from two Samsung (different models) cameras have identical quantization tables. Individual values of high frequency in those quantization tables are different from the corresponding ones in the closest standard quantization tables. Recompression operator of MATLAB cannot detect the in-camera compression of those quantization tables. Other images using standard quantization tables from the two cameras can be correctly detected. In addition, we examined several DSLR (digital single lens reflex) cameras of SONY make. All of those cameras provide only one compression quality. Similarly, they have custom adaptive quantization tables and we cannot detect their in-camera compression through recompression operator of MATLAB. In our experiments, we gave up the images from those DSLR cameras. However, we can resolve the problem of non-standard quantization tables via defining finer levels of quantization tables. We can combine those custom adaptive quantization tables into standard tables. This means we must construct a finer recompression sequence for detecting the in-camera compression.

Universality

We have drawn a conclusion based on our experimental analysis above: The earlier a compression is implemented, the larger the corresponding local minimum on the d q curves is. As mentioned earlier, this conclusion can be generalized to multi-compression. Hence, the proposed method can be applied to multi-compression history detection and in-camera compression detection in that situation. With regard to universality, another problem that has to be taken into consideration is the effect of using different out-camera compression tools with non-IJG quantization tables, e.g., Adobe Photoshop. If an image has undergone out-camera compression of Adobe Photoshop, which, as mentioned before, has its own quantization tables, the in-camera compression still can be detected by MATLAB as long as standard quantization tables was used by the camera. In this case, we first compress the test image with the standard quantization tables at QF=100. Then the image is detected using our proposed method. Figs. 9 and 10 show the d q curves of two doubly compressed camera images. Both of the two images have been doubly compressed in Adobe Photoshop at QF=8 and QF10, respectively. The in-camera compression can be explicitly revealed by the visible performance on the three d q curves. In summary, the proposed method is robust to multi-compression and different sources of out-camera compression. It has universality.

Fig. 9
figure 9

The luminance and chrominance d q curves of an image doubly compressed with Adobe Photoshop. The image is taken by NIKON E7900. The primary quality factor is 68. Recompression operated is performed by MATLAB

Full size image
Fig. 10
figure 10

The luminance and chrominance d q curves of an image doubly compressed with Adobe Photoshop. The image is taken by Samsung Techwin. The primary quality factor is 81. Recompression operated is performed by MATLAB

Full size image

Image spatial complexity

We have discussed the effect of image content on the performance of d q curves in Section 3.2. In our experiments, we have used different sizes of image blocks. For smaller-sized blocks, the image spatial complexity has to be taken into consideration to avoid uniform blocks. We have employed block variance to measure the intensity change of a block. In our experiments, the threshold of block variance is set to 10. In practice, a natural image as a whole must have a certain degree of intensity change. Hence, the proposed method will not be significantly affected by the image spatial complexity if the size of image block is large enough.

Limitations

Limitations of the proposed detection method are discussed here. First, the in-camera will not be presented on the luminance d q curve if the image undergoes an out-camera compression with a quality factor lower than the primary. That is, compressing an image at a lower quality factor (even with a common tool) than the previous compression will mask the earlier compression with a higher quality factor. However, we can conform that the image has undergone an out-camera compression because of the absence of the in-camera compression. This information is also useful for image forensics. Second, a difference between two compression qualities, i.e., the first and the second compression quality, is required. In our experiments, the difference is 2 at least. Third, the proposed method may be attacked by misaligned JPEG compression. But if the misalignment attacks the detection, we can shift the image by 0 to 7 pixels in the horizontal and vertical directions as did in [6]. Once the misalignment is eliminated, d q curves will present the compression history information.

5 Conclusion

In this paper, we have proposed a method of identifying in-camera compression from out-camera compression for a single image. In our method, we detect the in-camera compression through if the d q curves of the chrominance components do not present minimum in good coincidence with the luminance curve. The performance of the d q curves is attributed to the transition of subsampling type from the camera to MATLAB. Existing methods of detecting double JPEG compression can identify doubly from singly compressed images, and some can estimate the quality factor or the quantization table of the primary compression. As far as we know, no method addressing in-camera compression detection has been reported in the literature. Compared with the previous works, the advantages of this approach are outstanding: It can not only correctly identify the in-camera JPEG compression, but detect the corresponding quality factor; It does not need the estimation of primary quantization table by applying machine learning as did in previous works; The proposed method works well even when it is performed on small size of image blocks. This means it is also a promising method to image forgery detection and location; Our conclusions and proposed method can be extended to multi-compression history detection; The proposed method is robust to different sources of out-camera compression, e.g. Adobe Photoshop.

The detection of in-camera compression will be helpful to improve the image manipulation standard in journalism or in a photography competition. Our future work is constructing universal set of quantization tables including standard and Non-standard to make the system more practical.