Keywords

1 Introduction

In today’s world, computer and other devices like smart phones are in use in a great number which also increase their involvement in the crimes. Hackers and fraudsters are finding these as easy tools for committing the crime. Digital forensics or computer forensics tries to search the evidence from those devices so criminals can be charged in a court in a legal manner. Forensics is a significant service in maintaining law and order situation. In a forensics process, involved device is taken and a full copy of the device hard disk is taken for further analysis. There is no doubt of the good work that is done by the computer forensics but there are many files which a forensics team get that are not relevant for the investigation. Another serious privacy matter arises that when there comes the chance that an investigated person may not be turn up as a convict, compromising his privacy in vane. Any file can reveal the privacy of a person but none more than image files that can be very personal and reveal of them causing a serious privacy breach scenario. Now, people are very aware and conscious about their privacy and even don’t want authorities to get involved in it.

Images can be very helpful not only prosecuting someone for the possession of illegal image but can also identify a person involvement in some very illegal businesses like arms trafficking and child pornography. Digital image forensics is a vast field working in identification of source of an image with the help of image features, individual source identification and identification of modifications. We can also say that image forensics involves photogrammetry, photographic comparison, content analysis and image authentication. Our focus is on photographic comparison and content analysis of the image. The scenario that we will be going to follow is of keeping images of guns that can indicate the possession of illegal images, guns trafficking or though a slight chance but can also prove involvement in a murder.

Currently image privacy and security is preserved with the means of two techniques, one is encryption of image into a meaningless form and second one is scrambling which just scramble around the pixels of image so a human eye can not identify the contents of the image. Scrambling is a lot better than encryption in way of extracting features. Arnold’s Transform [1] is a widely used technique of scrambling and known for preserving features of image and its periodicity making image recoverable again. Features extraction can be done with methods like Scale Invariant Feature Transform (SIFT) [2] and Oriented Fast Rotated Brief (ORB) [3] which combined with Support Vector Machines (SVM) can provide great results.

We have combined all these different powerful techniques to make our proposed technique for privacy preserving image forensics. Our approach is better than the perceptual hashes [4] used before for providing privacy preserving image forensics because of it’s robustness & accuracy along with the preservation of security & privacy as well. We will explain our technique in detail in upcoming sections but one key thing to note is that with the help of this, crimes given above can be investigated in a fast and easy way. Our main contributions in this work which are summarized as follow:

  • We have improved the already present Arnold’s Transform (AT) algorithm which gives privacy but is not very good in terms of security which can lead to the breach of privacy. We bounded it with a key that can give it much better security and we have also increased it’s efficiency by modifying it for our approach.

  • We have created a new approach from a combination of AT algorithm, ORB and SVM to provide an efficient and privacy preserving method for the image forensics.

  • We have explained our technique working according to the forensics process and also provided some privacy policies that should be implemented along with our technique.

  • We have just not given a theoretical view but also proved with the help of results that are generated by our experiments.

We have given our related work in Sect. 2 and then provided detail of Enhanced-AT(EAT) in Sect. 3. Our proposed technique is explained in Sect. 4, forensics process combination with privacy policies in Sect. 4.2 and then results & experiments in Sect. 5. Finally, we have concluded our paper in Sect. 6.

2 Related Work

Digital image forensics is an important part of digital forensics that helps in gathering evidence from the images with the help of different tools and is connected with other domains in computer sciences. We have included a detailed related work because the combination of different fields. We have provided related work in our related fields which also shows the importance of our work.

2.1 Privacy in Digital Forensics

Privacy in the digital forensics is a quite interesting topic and some researchers have worked on. In [5] security and privacy is discussed in the computer forensics context. Privacy violation can be here that forensics investigator has access to even all the unrelated files too when a copy of whole device is made. There are different policies given for the investigators that should be followed, one of which is removal of unnecessary data. Another study shows that only policies are not enough for this purpose so in [6] a study was done on the privacy problems in the computer forensic and gives that the collection of only related data should be done like only emails that are related to the topic not thousands of others too. Already published digital forensics investigation models and procedures are not supporting the data privacy protection. Fourth amendment and many other privacy policies are already supporting the privacy but encryption should also be done to protect data in both collection and analysis phase. Here, AES and HMAC like stream ciphers are recommended because they are fast and efficient.

As above is general discussion about the privacy in digital forensics but now there are already a technique present that is used for privacy preserving image forensics in a practical way given in [4] as an architecture for image recognition and maintaining privacy. Two departments of law enforcement agency are working together to catch the people with illegal content. First department starts the investigation by encrypting the collected images and after that gives to the other department which match the extracted perceptual hash values from collected images with the already present database of hash values.

Table 1. A comparison of SIFT (Accuracy 58.28% time = 0.153), ORB (Accuracy 51.95% time = 0.024) and SURF (Accuracy 50.43% time = 0.049) on differently distorted and rotated images
Full size table

2.2 Image Encryption (Scramble) Methods

There is not much study until now in the privacy preserving image matching and forensic but there are many image scrambling techniques that are present. The most used is AT or Arnold Cat map first presented in 1960s, image scrambling technology depends on the information hiding technology which provides the techniques of information that don’t need passwords. Arnold transform is widely used because of its features like simplicity and periodicity. Arnold transform was only applicable for squared images when it came out in 1960s as Arnolds cat map but in [1] an algorithm for non-square images is provided. There are also phase scrambling technique given in [7] that scrambles images according to the phase co-relation only and match them without any key needed, key is only one time used for scrambling. In [8] there is a comprehensive survey on different image scrambling techniques which includes Rubiks Cubic algorithm, Arnold Transformation, Sudoku Puzzle scrambling and R-Prime shuffle technique. All these techniques can scramble image and then unscramble it easily. Similarly, in [9] different scrambling techniques are presented including Arnolds Transform, Poker Shuffling Transform and Fibonacci Transform. In all papers AT is found very efficient and providing enough security.

2.3 Feature Extraction and Matching Methods

Extracted features can provide very powerful image matching and retrieval in the domain of image processing. This can also be extended to next level of finding the features from scrambled domain that match our requirements. There are many methods but some of most popular of them are SIFT which was first published in 1999 and patented [2]. It uses differences of Gaussians (DoG) and extract local features, in [10] a privacy preserving SIFT (PPSIFT) concept is given which uses a homomorphic encryption technique and do DoG in Paillier encrypted domain. Speeded-Up Robust Features is another technique given in [11] is also patented and it follows same principles and steps like SIFT but is different like it uses square shaped filters as approximation of Gaussian smoothing for detection. Latest one is Oriented FAST and Rotated BRIEF (ORB) developed by Open CV labs and is totally free to use and is much faster than previous techniques as given in [3]. It is a mixture of two techniques FAST (Features from Accelerated Segmentation Test) used to find key points while BRIEF (Binary robust independent features) for visual descriptors. In a latest study [12] there is a comparison of SIFT, Speed Up Robust Feature (SURF) and ORB performance of image matching in a distorted domain. We have combined all in Table 1 to show a combined performance of all three. When machine learning like SVM are applied on feature extraction methods, results can be very good.

3 Enhanced Arnold Transform

Arnold’s transform is a key element in the working of this technique because of its ability to provide privacy along with homomorphic making operations possible on scrambled image. Introduced as Arnold’s cat map by Vladimir Arnold in 1960, it works with a square image. First the image is sheared one unit up, then two units to the right, and all that lies outside unit square is shifted back by the unit until it is within the square. This is shown in equation below:

$$\begin{aligned} T = \left( x, y \right) \rightarrow \left( 2x+y, x+y \right) mod\, 1 \end{aligned}$$
(1)

AT is better than other scrambling techniques because of its simple mathematical operations usage and a controlled image scrambling process. It can be seen in (1), how it works. An image de-scrambling can be done in same way as scrambling, running iterations again. When first introduced, it could only be used with square images which was a weak point for it but then in [4] a method for non-square images was proposed. In non-square approach, same square algorithm is used but image is divided into multiple square and each square is scrambled separately. On the other hand, for the purpose of descrambling all squares in the image are descrambled to get the original image.

figure a

In Algorithm 1 we have given our method that is used in our proposed technique for scrambling. This algorithm needs an image, a key converted into an integer value and iterations which is also an integer value that tells algorithm that how much scrambling to image should be done. The key here is given to the AT to control it in a secure manner. It can be seen in (1) that a value 2 is given to x and y, instead of 2 here is user choice represented as a key. In this transform to properly work in descrambling again a full same equation is required, so after some iterations original image can be regenerated. So, without the key it is really hard to descramble the image, though a word of caution that very high values of key can scramble image very much and can hide much of the information. Key can have values like 2, 10 or any integer, if some one tries to descramble image without the key he has to use brute force to try each value and also has to wait for many iterations to check if this is the image or not making it a two layer security. Now come to the iterations value which is also user choice and also controls that how much scrambling should be done because if equation runs again and again it will keep scrambling the image and will produce very scrambled image. We called it EAT because in original one there was no control over security and efficiency but now it can be controlled. Privacy is given by transform because images become enough scrambled that a human eye can’t recognize what is in the image. Security is provided with two things, first one are iterations that are needed to descramble images and second one is now the key which we have provided as a new security measure. The great thing about this scheme is even from the noisy looking image, features can be extracted which we are going to discuss in upcoming section of our proposed scheme. One thing to be noted that if security is high getting features from images becomes difficult, so there is a balance needed between security and usability. We have explained it with results in the results and experiments section making it more understandable.

As for the non square approach our same algorithm will be used but to scrambled each square of the image has to be scramble separately and than rejoined to a single image again. For descrambing, each square inside the image has to be descrambled and then the combined image of all the squares will show original image. It is a lot more complex and slow than the original AT because of taking parts of an image and applying algorithm on them. Because of this complexity it is also more secure than the non-square technique. For readers more interested in AT we recommend [1] and [13].

4 A New Approach for Privacy Preservation

We have created our approach from a combination of already state of the art techniques in this field. There is not much work has been done on the privacy preserving computer forensics except defining different policies. We have chosen SVM, AT and ORB from machine learning, encryption & scrambling domain and feature extraction respectively. We have already explained in detail our EAT so first we are going to provide a little detail about ORB and Support Vector Machine and then we will give our proposed scheme.

4.1 Preliminaries

Support Vector Machine. SVM is a very popular and used machine learning approach like decision trees & neural networks. SVM is used for both classification and regression like other present techniques but its real power is its kernel system. It works by taking a training set which is divided into different categories, SVM algorithm assigns them into different categories. Basically SVM model is representation of examples as points in space, these points are mapped so that separate categories are divided with a clear gap as wide as possible [14]. It can also perform non-linear classification with the help of kernel trick and represent them in a higher dimensional space. With the use of kernels best learning parameters can be found for given data. In case of linear kernel or no kernel boundary will be separated with a straight boundary and requires more samples for better accuracy. In a kernel approach boundary can also non linear like circular or even more irregular one and requires less samples for producing same accuracy as a linear one. Radial Based Kernel or RBF is commonly used in SVM and have two parameter one is called as gamma and other one C. Gamma can be thought as spread of the kernel, If gamma is low than the decision boundary will be broad but when gamma is high decision boundary is low. On the other hand C is a penalty parameter, if low classifier will be okay with the incorrect classifications and then other way around. We have chosen technique because of its efficient performance and ability to handle different type of data especially image features in the form of visual bag of words.

Oriented FAST and Rotated BRIEF. Oriented FAST and Rotated BRIEF simple called as ORB [3] is a feature extraction and description technique. Like its predecessors SIFT and SURF it performs same functionality but in a fast and accurate manner. It uses FAST for the purpose of detection of keypoints, when keypoints are found Harris corner measure method is applied to find top points among those points. One problem which was there that FAST is not compute the orientation of the points, so authors used intensity weighted centroid computation for the patch that have corner located at center. So, now the direction from the corner point to centroid gives orientation. On the next stage are descriptors used for extracted features description, ORB uses BRIEF descriptors but it was performing poorly with the rotation, so for solving this problem it is steered according to the orientation of keypoints. This is much faster than SIFT [2] because of not using Difference of Gaussian which is very slow and uses all the points which are in most cases are unnecessary. In Fig. 1, we have compared SURF, SIFT & ORB which shows that ORB is very fast as compare to SIFT and is very little less in accuracy than SIFT. Accuracy of ORB is increased a lot with the use of SVM and due to its robustness it will be helpful in analyzing a big number of images in a little time. For readers interested in more detailed working of ORB we recommend [3].

Fig. 1.
figure 1

A system model explaining the technique

Full size image

4.2 Proposed Scheme

Now, lets come to the scheme that we have proposed for solving the privacy issue in image forensics. We have used EAT for scrambling purposes so privacy and security is ensured. Then we have used SVM to train classifier with the features extracted from the scrambled images and then different scrambled images are given for test that if there is an image present with same features can be detected. We have explained working of our whole system with the help of a system model and then provided its properties as follow:

System Model. Our system model explains the working of our scheme and consist an ORB extractor, a svm classifier and features matching mechanism Fig. 1. It works by first choosing a dataset on the basis of what type of images we want to detect from a computer like guns, cars or like. Images are than scrambled and given to a SVM classifier which can has a linear kernel if dataset is large and so samples will be greater. But in case of a dataset with a low number of images a RBF or other kernels can be used to get better results and a trained classifier is produced. On the other hand, the images that we are going to test whether they are illegal or not are also in scrambled form protected by a key which can be known to forensic team leader or with other third party who can be considered as neutral and cannot be cause of privacy breach. These scrambled images are analyzed with the help of already present trained classifier and results are checked that is there a positive match or not.

If the match is negative than the process can be ended or images can be checked again and again, may be a positive result could come latter. If found nothing than the process can be ended but if there is a positive match than image will be descrambled. For the purpose of descrambling, key that was used in the process to scramble images will be needed and retrieved images are checked to be sure by a human analyst that machine doesn’t produced a false positive. Now this is interesting here this is kind of double check for identifying the illegal image, how much trained a machine can produce false positives but human eye can confirm whether this is correct or not making this technique strong. This also increases the value of evidence in the court because it is not just a machine given results and also checked by human eye. It also remove the problem that privacy of an innocent of crime person privacy can be breached.

System Properties. Our technique has made possible providing privacy in image forensics in a practical way rather than just relying on the policies [6]. A privacy preserving technique in forensics should provide privacy of course, then efficiency and also security so this can not become the cause of privacy breach, our technique has all these properties making it good for using in image forensics while keeping privacy of a person intact. These properties are explained as follow:

  1. 1.

    Privacy: It is of course foremost property that a privacy preserving system should have and privacy is provided in our technique in two ways. First one is with the help of scrambled images which are can not be identifiable by an human eye. Secondly, we have used SVM which is a machine learning technique and unlike an human analyst, machine analyzing the images can not be considered as privacy breach.

  2. 2.

    Security: Another powerful aspect of our technique is making sure provision of security which also increases privacy. We have developed an EAT given in Sect. 3 that provides a double security. At first it is bind with a key which is if not given images can’t be descrambled without brute force. But there is a second security measure in AT that many iterations are required to get back an original image [1]. So, attacker has to use brute force for key checking and also have to check image all iterations that original image has been produced or not, making it a lot more difficult.

  3. 3.

    Efficiency: All three techniques that combines into our technique are very fast in scrambling, features description and machine learning. This also gives a boost to our technique making it very fast. SVM and ORB combination increases the accuracy of feature matching. So, in few word it can be said that it is very efficient in both accuracy and robustness.

  4. 4.

    Double Check: Our approach also provides a double check mechanism, first machine match images but its results can be false positives. So, instead of just relying on it a human can check the images that came as positive. Use of this ends the problem that someone can be falsely accused and also ensures privacy that only images which are flagged as positive are checked not all collected images (Fig. 2).

Fig. 2.
figure 2

Results showing our approach outputs with labels on left and on right unscrambled form of images

Full size image

5 Experiments and Results

5.1 Experimental Conditions

For the purpose of checking the performance of the scheme we have applied it on different datasets which contain different number of images. The good property of our technique is that with use of ORB it can produce great results with small amount of data. We have created dataset from pistol as one category and other one as other which contains different images like vehicles including aeroplane, cars and bicycles. We did this because we only need to obtain some pistol results not other images present in the user device because this can breach the privacy. We have taken datasets from just 30 images for other and pistol category to 500 images for each category. ORB needs just few total clear images of the element to be found and it produces rather improved results with a combination of SVM Fig. 5.

In our practical implementation, we have used Python for the purpose of image processing and machine learning. We used libraries like sci-kit learn, OpenCV, matplotlib and numpy for the purpose of machine learning and image processing. System that we used has core i7 2.4 GHZ processing chip, 16 GB RAM and a 4 GB graphical chip.

Fig. 3.
figure 3

Images scrambled to iteration 3 of the Arnold Transform

Full size image

5.2 Image Scrambling

We have used our EAT for the scrambling purposes, we will explain our most successful dataset which was consist of 102 images total and with a dimension of 200 px. When all 102 images scrambled to 3 iterations, this whole process took only 3.30 sec and provided results like Fig. 3.

It can be seen that with iteration 3 scrambling a lot of prominent feature are there which can be extracted with the help of ORB. But when we scrambled all images to 4 iterations, time taken was 18.54 s and example of this scrambling is shown in Fig. 4. Along with much time taken there are less prominent features in this much scrambled image. Basically this depends on how much image is scrambled, if image is scrambled too much there will be small number of prominent features to be extracted. On the other hand scrambled image at 3 or 4 iterations can produce some good results.

Fig. 4.
figure 4

Images scrambled to iteration 4 of the Arnold Transform

Full size image

5.3 Final Results

As above some results of scrambling are shown, here we will talk about our final results where SVM and CNN is applied on the scrambled images given for test. With these experiment we concluded that if we will increase the scrambling of images it will be more difficult to obtain good features from the images. By applying CNN, gives unfair results of descrambled image. When SVM is applied all results that come up with label pistol saved in a other folder. We descrambled images through SVM can be easily as compared with CNN come up with label pistol by going to some iterations and got original image back, these results are shown in Fig. 6 where it can be seen clearly that scrambled images are labeled with pistol and other by SVM which saved the results after running the classifier on scrambled images given for testing. On the other side there are original images which shows that in other is a bicycle and car as other category contained aeroplanes, bicycles and cars. Similarly, there is also pistol descrambled image which represent the second training category pistol as an example of gun. This also has given very fast results and test 11 images in 0.2 sec while training took only 4 sec which is pretty fast time and shows that how thousands of images can be test in a very little time.

Fig. 5.
figure 5

Accuracy results for SVM along with 500 data and CNN along with 102 data

Full size image
Fig. 6.
figure 6

Confusion Matrix showing accuracy of our approach

Full size image

We used different datasets and even with 500 images we got only upto 96% of accuracy through SVM but with only 102 clear images got accuracy more than 80% through CNN which is a good accuracy percentage. When images scrambled to iteration 3 we have got very promising results as more than 96% accuracy but in case of iteration 4 scrambled images we could only get up to 70% accuracy point which is a low accuracy point. We have shown our accuracy in the form of a confusion matrix in Fig. 6. Here the diagonal elements represent the number of points which are predicted, truly while elements off the diagonal represents the elements that are misplaced by the SVM classifier. Figure 5 show the graphical view of comparison between CNN and SVM classifiers in term of accuracy achieved by them.

6 Conclusion

We have proved that it is possible to preserve privacy in a practical manner and with automation becomes more efficient. We have given a full model of our proposed technique that how it works and whole forensics process explanation based on a scenario which shows that this is really applicable in practical use. There are also some policies for our technique are defined which can protect user privacy in a much better form. All in one our technique is a full framework that can be used for the purposes of identification of illegal images with preserving user privacy. This can provide very fast results which can help in fast progression of cases like guns trafficking and child pornography. But it is very difficult to provide privacy in computer forensics investigations unless a persons that are responsible for implementing this technique follow moral values. But it is a lot better than just implementing policies for the investigators because it is a solid approach. We have basically showed that it is possible to provide privacy in computer forensics in a practical way.