Detection of Tampering by Image Resizing Using Local Tchebichef Moments

Abstract

There are many image resizing techniques, which include scaling, scale-and-stretch, seam carving, and so on. They have their own advantages and are suitable for different application scenarios. Therefore, a universal detection of tampering by image resizing is more practical. By preliminary experiments, we found that no matter which image resizing technique is adopted, it will destroy local texture and spatial correlations among adjacent pixels to some extent. Due to the excellent performance of local Tchebichef moments (LTM) in texture classification, we are motivated to present a detection method of tampering by image resizing using LTM in this paper. The tampered images are obtained by removing the pixels from original images using image resizing (scaling, scale-and-stretch and seam carving). Firstly, the residual is obtained by image pre-processing. Then, the histogram features of LTM are extracted from the residual. Finally, an error-correcting output code strategy is adopted by ensemble learning, which turns a multi-class classification problem into binary classification sub-problems. Experimental results show that the proposed approach can obtain an acceptable detection accuracies for the three content-aware image re-targeting techniques.

1. Introduction

As image editing tools and various mobile devices are easily acquired and conveniently used, maximizing the viewing experience of end users on small devices becomes very important. Compared to traditional image re-targeting methods, such as linear scaling and cropping, many content-aware image resizing methods can preserve salient areas, avoiding serious distortions or loss of significant information [1,2,3]. Meanwhile, many content-aware resizing algorithms have been adopted using image editing tools, such as photoshop and GIMP. An ordinary user can very easily create tampered images for malicious purposes using image editing tools. Moreover, it is impossible to distinguish those tampered images from authentic images with the naked eye. Therefore, how to detect tampered images is a hot topic in the field of image content security.

In recent years, a few approaches have existed about the detection of content-aware image re-targeting. Moreover, most of the detection methods are for the seam carved images. Lu et al. adopted a forensic hash to distinguish whether an image is subjected to a seam carving operation [4]. However, it is an active forensics approach; moreover, a falsifier might remove the forensic hash. For passive forensic detection, Sarkar et al. used 324D Markov features to detect image seam carving [5]. Later, Fillion et al. exploited a series of intrinsic features to expose the trace of seam carved images [6]. In Wei et al. [7], an approach based patch analysis was adopted to distinguish whether an image is original or not. According to noise and energy distribution in seam carved images, Ryu et al. [8] exploited the features based on noise and energy bias to detect seam carved images. Local binary pattern (LBP) was adopted to detect seam carved images [9] in our recent work. Inspired by image entropy with the ability of capturing the intrinsic information of an image, we exploited multi-scale spectral and spatial entropies to detect seam carved images with low resizing ratios [10]. Web Local Descriptor (WLD) and LBP were adopted to distinguish whether an image is original or not [11]. In [12], a large feature mining approach was proposed to detect image seam carving under recompression in joint photographic experts group (JPEG) images.

However, most existing detections of image resizing are designed for a specific content-aware resizing. Much less has been done to distinguish different content-aware resizing approaches in the process of image re-targeting. In practice, the best re-targeting method relies on an image itself. For example, scaling images in horizontal or vertical direction can be performed in real time using interpolation and will preserve the global visual effects and retarget images with medium perceptual quality. However, scaling will introduce some shape deformation into the retarget image. Seam carving [1] supports various visual saliency measures for defining the energy of an image. Nevertheless, seam carving can excessively carve less important parts of an image and result in unwanted visual distortions. scale-and-stretch [3] can preserve the aspect ratios of local objects. However, if there are many quads in the image, the approach will fail to preserve the aspect ratio of the whole image [2]. Therefore, there are different image resizing methods depending on the image content to achieve image change in size while preserving the saliency region. it is necessary to propose a universal detection of image resizing.

The rest paper is organized as follows: Section 2 summarizes several common methods of image re-targeting and analyzes their artifacts. Section 3 briefly introduces the proposed detection approach. Our experimental results are described and analyzed in detail in Section 4, and conclusions are made in Section 5.

2. Image Resizing and Their Possible Artifacts

2.1. Several Common Methods of Image Resizing

Among the methods of content-aware image resizing, scaling, seam carving [1], and scale-and-stretch [3] are three common approaches to re-target an image. Seam carving is defined by forward energy. The intensity gradient magnitude in $L_1$ metric is defined as an importance map. The contiguous chains of pixels that pass through the regions of the least importance in the image are deleted or duplicated by seam carving to obtain image resizing. Dynamic programming is adopted to compute seams. Scale-and-stretch is defined warping. Both image dimensions are processed by warping at once. Moreover, an objective function is optimized to allow important regions uniformly scaling in order to preserve their shapes. A saliency and combination of $L_2$ gradient magnitude (defined by [13]) is defined as the importance map. Scaling implemented image resizing by simply bi-cubic interpolation and non-uniform scaling.

According to the above description of the three resizing methods, it is clearly found that scale-and-stretch can keep significant regions in an image, which is consistent with its original image after the image is re-targeted. However, seam carving implements image re-targeting by deleting or inserting pixels within a minimal energy; therefore, it may cause the salient object distortion.

2.2. Analysis of Image Resizing Artifacts

There exist three kinds of artifacts for the image processed by the content-aware resizing method [13], such as geometric deformation, information loss and local texture distortion. Figure 1 shows these artifacts caused by content-aware re-targeting. Figure 1b shows line or edge distortion after re-targeting. However, salient areas of an image, such as people and building, do not significantly change. The shape distortion of an image is shown in Figure 1c. This further explains that removed pixels might exist in salient areas of the image when a pixel with a minimum energy is deleted in the process of the image seam carving. Therefore, it is easy to deform important objects of the image in the process of image resizing. Figure 1d shows the artifact of information loss. A scaling method is bi-cubic interpolation and re-targets the entire image in the process of image re-targeting. To better show the image distortion in the process of image resizing, we adopt an LTM histogram to identify the distortion caused by different resizing methods in this paper. Figure 2 shows the residual LTM diagram for three different resizing methods. It can be found from Figure 2b that the image distortions of a non-subject area are not easily perceived in Figure 1b. However, these distortions can be clearly found in the residual LTM diagram. It can be clearly showed that the distortions are found in the process of image re-targeting in Figure 2d.

3. Proposed Method

A passive detection method is presented for image resizing forgery detection in this paper. Figure 3 shows the implementation process diagram of our presented algorithm. Similar to the process of most existing forensics methods, our proposed method consists of two parts, i.e., a training part and a testing part. In the training process, tampered images and their corresponding original images are adopted as data sets. First, all the training images are preprocessed. Second, the LTM histogram features are extracted from preprocessed images. Finally, a training model is obtained by using ensemble learning based on extracted features. In the testing process, the LTM histogram features are extracted according to the same steps in the training part. Finally, the extracted features are used by the trained ensemble classifier to distinguish which resizing method a tested image is re-targeted.

3.1. Preprocessing

An image obtained by content-aware resizing methods usually has a good visual effect. Furthermore, it is impossible for users to distinguish from authentic photographs using the naked eye. However, the correlation between adjacent pixels will inevitably change after an image is resized. Therefore, it is necessary to preprocess re-targeted images. Image residuals can efficiently capture the change of adjacent pixels in the process of image re-targeting. In this paper, a one-dimensional low-pass filter is adopted to calculate residuals along the horizontal and the vertical directions. The formula is shown as Equation (1):

$$R(x,y)=I(x,y)-I(x,y)\ast L\left(u\right),$$

(1)

where $I(x,y)$ is an image and $L\left(u\right)$ is low-pass filter. Figure 4 shows the residuals of the preprocessed content-aware image resizing. Through the residuals, it can be clearly found that tampered traces are caused by different content-aware image resizing methods.

3.2. Features of LTM

After images preprocessing, orthogonal Tchebichef moments are adopted to construct feature vectors on 5 × 5 neighbor pixels. In addition, the texture information is encoded with Lehmer to represent the relative strength of moments. The extracted feature vectors are called LTM. A byte value for each pixel is provided, and an LTM diagram is generated by the encoding scheme. Therefore, the histogram features of LTM are adopted to identify whether an image is subjected to image resizing. Figure 5 shows the histogram features of LTM after preprocessing.

3.3. Ensemble Learning for Blind Forensics

In this paper, an error-correcting output codes (ECOC) strategy [14] based on ensemble learning is adopted, which transforms multi-class classification problems into binary classification sub-problems. This is because ECOC is an excellent multi-class classification tool, and the ensemble learning performs well in terms of computational complexity and detection accuracy. The tamper of three different resizing methods, such as scale-and-stretch, seam carving and scaling, is identified. Therefore, for this three-class classification problem, a pair coupling strategy [15] is adopted. Specifically, a discrete matrix (coding matrix) is defined first. In addition, then, the problem is decomposed into n = 3 binary classification sub-problems according to the sequence of 0 and 1 in the coding matrix, namely dichotomies. After that, ensemble learning is adopted to train these dichotomies and test the extracted histogram of LTM to obtain binary vectors. Finally, the class is identified by the minimum hamming distance between the encoded word and the vector.

4. Results

4.1. Experimental Environment

To verify the performance of our proposed algorithm, we conduct a number of experiments in our personal computer. The passive forensics approach is implemented in Matlab2012b. The ensemble learning can be downloaded from [16]. In this paper, the Uncompressed Colour Image Database (UCID) [17] is adopted as the original images. The image database contains 1338 images, which are composed of people, buildings, animals and landscapes. Since there is no publicly available image database of image resizing available, we construct an image library from UCID for resizing carving detection. According to different resizing ratios, three resizing methods are used to produce tampered images. The resizing ratios vary from 10% to 50% with a step size of 10%. That is, for every resizing method, the resizing ratios of tampered images are 10%, 20%, 30%, 40%, and 50%. Therefore, we have 1338 original images and 3 × 5 × 1338 tampered images. To verify our proposed approach, we perform the performance evaluation for the following cases: (1) tamper detection for a single resizing method; (2) tamper detection for multiple resizing methods; and (3) tamper detection without preprocessing. In all experiments, the ECOC based on an ensemble learning strategy is adopted to test the effect of our proposed method. The image dataset is divided into two groups, 50% for training and 50% for testing. The training and testing are repeated ten times, and the average detection accuracy is reported in this paper.

4.2. Experimental Discussions

4.2.1. Tamper Forensics on a Single Resizing Method

In this experiment, we test the detection performance of our proposed method for a single resizing method. The tampered images with scaling ratios from 10% to 50% are adopted to test the performance of our proposed approach.

Table 1. Comparisons in terms of accuracy for tampered images with single re-targeting methods.

Scaling Ratio (%)	Accuracy (%)
	Scale-and-Stretch	Seam Carving	Scaling
10	82.51	75.26	74.33
20	88.45	91.67	87.37
30	95.51	97.87	95.55
40	98.95	99.77	98.87
50	99.85	100	99.81

shows the detection results. From Table 1, we can see that the detection accuracy is improved with the increment of the scaling ratio. When the scaling ratio is less than 20%, our proposed approach has a higher detection accuracy for the scale-and-stretch resizing method. Since the optimal local scaling ratio of each local block is calculated iteratively and the warped image is updated simultaneously to match the scaling ratios as much as possible, the entire image is resized in the process of scale-and-stretch re-targeting. However, the seam carving method resizes an image by deleting the seams with the lowest energy once. Therefore, the tampered images obtained by the seam carving method are difficult to be distinguished from the authentic images when the scaling ratio is low. With the increment of the scaling ratio, the algorithm will cause a global structure distortion. It can also be reflected in Table 1. Our proposed approach can get a higher accuracy rate for images obtained using the seam carving method than that using other resizing methods with the increment of the resizing ratio.

4.2.2. Identifying Images Obtained by Different Content-Aware Resizing Methods

In this experiment, the tampered images with scaling ratios from 10% to 50% are obtained by different content-aware resizing methods. They are adopted to test the performance of our proposed algorithm. The average detection accuracies of different content-aware re-targeting methods, where the average detection accuracy is the average value of diagonal elements in the confusion matrix, are summarized in

Table 2. The average detection accuracies of our proposed approach with preprocessing.

Uncompressed (%)						Compressed (%)
10%	20%	30%	40%	50%	mixed	10%	20%	30%	40%	50%	mixed
82.04	90.00	93.30	94.61	95.94	92.04	76	81.13	84.08	87.98	88.34	78.33

. Note that “mixed” represents the mixed test set of tampered images with the scaling ratios from 10% to 50%. There are three content-aware resizing methods in this paper. Therefore, this is a four-class classification problem (the original images as a special class), according to the ECOC strategy.

Table 2 shows that the average accuracy is improved with the increment of the scaling ratio. However, the detection accuracy is apparently decreased for highly compressed images with Quality factor (QF) being equal to 75. Through careful analysis of our experimental results, it is found that the main reason for the decrement of the detection accuracy in the compressed condition is that the traces of the tampered images are weakened when images are compressed. Therefore, the detection accuracy is decreased in this case. We have also completed the experiment of the “mixed” tampered uncompressed and compressed images and get the confusion matrix.

Table 3. CMOMTUI and CMOMTCI with preprocessing.

Uncompressed	Detection Accuracy (%)				Compressed	Detection Accuracy (%)
	OR	SNS	SC	SL		OR	SNS	SC	SL
OR	95.93	*	*	*	OR	93.45	*	*	*
SNS	*	90.14	11.12	3.61	SNS	*	80.69	11.75	10.49
SC	*	*	83.71	6.01	SC	*	1.49	68.43	18.77
SL	3.26	8.90	5.17	90.38	SL	5.56	17.82	19.82	70.73

shows our experimental results, where CMOMTUI represents the confusion matrix of “mixed” tampered uncompressed images, CMOMTCI represents the confusion matrix of “mixed” tampered compressed images, “*” represents the classified accuracy less than 1%, OR represents original images, SNS represents a scale-and-stretch method, SC represents a seam carving method, and SL represents a scaling method. From Table 3, we can see that our proposed method can get a high accuracy for the three content-aware resizing methods mentioned in this paper. However, it can’t obtain a good detection accuracy on the tampered images with JPEG compression. In addition, its false positive rate is relatively high for the seam carving method (SC) and the scaling method (SL).

4.2.3. The Detection Accuracy without Preprocessing

Table 4. The average detection accuracies of our proposed approach without preprocessing.

Uncompressed (%)						Compressed (%)
10%	20%	30%	40%	50%	mixed	10%	20%	30%	40%	50%	mixed
84.04	91.02	94.32	95.41	96.04	93.04	70.93	74.59	79.63	83.18	85.34	76.09

and

Table 5. CMOMTUI and CMOMTCI without preprocessing.

Uncompressed	Detection Accuracy (%)				Compressed	Detection Accuracy (%)
	OR	SNS	SC	SL		OR	SNS	SC	SL
OR	96.39	*	*	*	OR	87.98	*	*	*
SNS	*	91.54	9.21	3.21	SNS	*	77.25	16.29	12.49
SC	*	*	85.71	5.41	SC	1.35	1.73	65.14	17.07
SL	3.60	8.40	5.08	91.38	SL	10.67	21.02	18.57	70.73

report the identified results for the uncompressed and compressed tampered images without preprocessing, respectively, where CMOMTUI represents the confusion matrix of “mixed” tampered uncompressed images, CMOMTCI represents the confusion matrix of “mixed” tampered compressed images, “*” represents the classified accuracy less than 1%, OR represents original images, SNS represents the scale-and-stretch method, SC represents the seam carving method, and SL represents the scaling method. It can be found from Table 4 and Table 5 that our proposed approach has a sightly higher detection accuracy on the uncompressed images when the features of LTM are extracted from the images without preprocessing. However, when the images are compressed by QF = 75, the accuracy of our proposed approach is significantly lower than that of the images with preprocessing. The main reason is that the residual may weaken the tampered trace of the images without compression when it is used in the process of preprocessing. However, the images with compression are preprocessed by residuals, and the changes of these images will be highlighted.

5. Conclusions

Content-aware image re-targeting methods are widely adopted to resize images to display on all kinds of terminals. However, they can be also used to make fake images, which don’t have any perceptual annoying distortions. By the principle analysis of the three image resizing methods, we found that the correlation between adjacent pixels can be destroyed in the process of image resizing. Tchebichef moments have been extensively applied in field of image/vedio such as information security [18], pattern recognition [19] and image quality assessment [20], and so on. Inspired by this, it can be found from experiments that LTM can effectively reflect the correlation changes between adjacent pixels. We proposed a passive forensics algorithm based on LTM to identify tampered images obtained by content-aware image resizing methods in this paper. Our experimental results showed that our proposed method can obtain a better accuracy. It is verified that it has a good performance for the image resizing with high scaling ratios. In the future, we will try to evaluate the detection accuracy on the tampered images obtained by image resizing with low scaling ratios. In addition, our proposed method could not obtain a satisfied detection accuracy on the re-targeted images with JPEG compression. We will further analyze the tampered trace of the resized images with JPEG compression. In addition, since there are still a few image resizing methods [21,22] besides the three methods proposed in this paper, we will attempt to distinguish these image resizing techniques for re-targeted images by applying other multi-class classifiers [23,24,25,26,27,28] and designing more general features from image/video processing methods [29,30,31,32,33,34,35,36,37,38]. In view of the importance of social media digital images in practical applications, research on their authenticity, integrity and traceability has been one of the hot and challenging research topics in the field of information security. We will adopt network optimization methods [39,40,41,42,43,44,45,46,47,48] to improve the real-time and high efficiency performance of the feature extraction phases.