1 Introduction

Three-dimensional printers have extensively been used in household manufacturing and techniques to protect copyrights are essential. Anyone can currently easily produce finished products by using their home 3-D printers. Customers may also prefer to create products that have high-end quality at low cost. Many people expect that such features of 3-D printers will change the ways in which manufacturing and physical distribution are carried out. However, this also means that pirated products can very easily be manufactured that have quality equivalent to that of regular versions. As the market for digital fabrication is certain to expand rapidly in the near future, it is clear that the piracy problem with 3-D fabrication will also become more serious. Thus, technologies to protect against piracy, such as technology to protect the copyrights of digital data for 3-D printers, are required so that the business of digital fabrication, such as that using 3-D printers, can grow.

Various techniques of protecting the copyrights of digital content have been studied thus far, including those for 3-D objects [1–3]. However, conventional technologies cannot be applied to protect digital content for 3-D printers because the final products here are real objects produced by consumers and the copyright of digital content should be checked from real objects after they have been produced, while conventional copyright protection of digital content is checked from digital data.

Since it is essential to check the copyright of digital content from real objects, we have proposed techniques where copyright information was embedded inside real objects fabricated with 3-D printers by structuring the inside cavities of objects. Moreover, we have proposed a technique that can nondestructively read out the embedded information from inside real objects by using thermography [4–7]. The basic concept underlying the techniques we have proposed is the same as that for conventional watermarking techniques, but it differs in that copyright information is embedded inside the real objects.

We experimentally demonstrated the feasibility of our proposed techniques in a previous study. We propose a new method in this study in which we use thermographic videos to read out copyright information, which is a technique that enables automatic processing in reading out embedded information.

2 Copyright Protection Technique Using Information Embedded in Inside of Physical Objects

Our proposed techniques can be used to check illegally produced real objects in which copyright information is concealed by nondestructively reading out embedded information using thermography. When the digital data for printing a 3-D model is created, the copyright information is contained in it so that when a real object is fabricated with a 3-D printer, it contains fine structures inside it that express the copyright information. These fine structures can be seen from outside the object. These internal fine structures are detected nondestructively and the copyright information can be read out. As such fine structures inside the object work as a kind of watermark, this technique effectively protects the copyright of digital data for 3-D printers.

Fine structures can be formed by making fine domains using materials whose physical characteristics differ from those of other regions inside the object. We can detect fine structures inside objects by using this difference in physical characteristics.

We formed fine cavities as fine structures in a previous study. Copyright information was represented by the characteristics of these fine cavities. Information was encoded in binary code and if there were cavities in predetermined positions, they were represented as ones or otherwise expressed as zeros.

We used a method of thermography to nondestructively read out information. If fine cavities are formed near an object’s surface, the temperatures of the surface area above the cavities become higher than those in other surface areas when the surface is heated because the cavities block thermal diffusion from the surface to inside the object because of the far lower thermal conductivity of the cavity regions than that of the other regions. Therefore, we can detect cavities inside the object from the thermal image of the surface that we can obtain with thermography.

We used still thermographic images in reading out the patterns of cavities in previous studies [4–7]. We found that heat irregularly spread out through the samples. It was difficult to capture only one image at certain times that could completely read out all embedded code. The technique we propose in this paper uses video processing to solve this problem. We recorded thermographic videos of a sample while heating and cooling it and we found that the code for the embedded cavities appeared and disappeared as partial areas over time. Therefore, if we could identify the codes in individual partial areas over time and then sum up all the images together, we could improve our technique of reading out information so that it became more effective.

3 Methodology

We embedded the information code patterns, represented by the fine structures, and read out the embedded code by recording thermographic videos. Patterns of fine structures were formed inside the object as a group of small cavities. After heating ceased, we recorded thermographic videos to measure the surface temperature distribution until the temperature had cooled down to the original temperature before heating. Finally, we used image processing techniques to detect the patterns of the cavities.

The sample, which was used in this experiment, was fabricated by using a fused deposition modeling 3-D printer with polylactic acid (PLA) plastic filament. The sample was prepared as a hard opaque black cuboid with a size of 5 × 5 × 1 cm (width × length × height). The pattern for the cavities is outlined in Fig. 1. The cavities were formed 1 mm under the surface. Figure 2 outlines the setup for the heating and video recording system that we used in this experiment. Two halogen lamps (maximum output of 500 W) were positioned on opposite sides to heat the sample. A thermal imaging camera (Testo 875) was placed in front of the sample to record the video and measure the distribution of surface temperature. The resolution of the recorded video was 160 × 120 pixels.

A video clip was recorded from when heating started until the sample was cooled down to room temperature (RT) in the process of image processing. The images were captured from the video in each frame. Background images were selected from when heating stopped to when the sample was cooled down and under condition:

$$ G = \frac{1}{{\sqrt {2\pi \sigma } }}e^{{ - \frac{1}{2}\left( {\frac{I - Bg}{\sigma }} \right)^{2} }} \approx \sigma 10^{ - 3} \pm 0.002 $$
(1)

where G is the index to select a suitable background scene (Bg), I is the original current image scene, and σ is the standard deviation of I or constant value (this value in the experiment was set at 15 as the best result). Note that the best registered background image should have the G index in this interval, σ10−3 ± 0.002.

Fig. 1.
figure 1

A cavities pattern schematic of ASCII code as “KAIT2014”. The gray square is where there is a cavity in the predetermined position; the white dotted line box indicates that there is no void in place. 0 represent as the cavity position and 1 is a cavity predetermined position. The size of each cavity as a 2 × 2 × 2 mm, the spacing between the cavity it was 2 mm.

Full size image
Fig. 2.
figure 2

A schematic of heating and recording system; two halogen lamps were placed far from the sample with 16 cm distance and set as 60° to the incident angle.

Full size image

The background image that was obtained contained the profile of luminance that referred to the ambient temperature when the sample began to be heated. It was used as the ground truth for each thermal video clip.

After that, the gray values in each image frame were subtracted from the registered background image to only segment regions that had temperatures higher than that in the ground truth. Then, small differences in illumination between surrounding areas and cavity patterns were compared to create binary images. If there were no differences, the value was set to zero and it was set to one if the region of interest (ROI) was a cavity. Binary images of cavity patterns in individual frames were obtained after processing. Finally, all binary images were summarized as all the video length to more clearly amplify the signal from cavities, as seen in Eq. (2). The cavity pattern in the proposed approach could be clearly detected as the pattern in Fig. 1.

$$ S = \sum\nolimits_{n = 1}^{N} {h_{n} } $$
(2)

Here, S is the summarized signal of the cavity patterns all frames of a video, h is the binary signal of each frame in a thermal video clip, and N is the number of all frames in a video clip.

4 Results and Discussion

The results for each step of image processing are presented in Figs. 3 and 4. Figure 3a has sample results for a video clip that found a suitable background image in frame number 123 as a registered background. The results for the background subtraction of a current scene from the registered background are presented in Fig. 3c.

Fig. 3.
figure 3

Results of finding registered background for background subtraction; (a) registered background, (b) current image, (c) an image result after background subtraction.

Full size image

Figure 4 presents the results for the signal that was obtained after processing when heating stopped after 1, 50, 100, 150, 200, and 300 frames. The first row has the original image of each frame. The second row has the signal for the cavity pattern after processing in each frame and the percentage of accuracy in recognition. The third row has the signal for the cavity pattern after processing with our proposed method that summarizes all frames of the video and compares the percentage of accuracy in recognition for each period. The binary images of the cavity patterns reveal white and black squares as a result. The white squares indicate that cavities were detected, and the black squares indicate that no cavities were detected.

The results indicate that the cavity pattern could be detected correctly as the accuracy was 100 % in the final image, but only in frame number 50, when we processed the pattern in each frame. However, when the images were processed with all images summed up together, i.e., the summed image from 150 frames and after, accuracy was 100 % in all the results. Therefore, these results demonstrated the efficiency of the technique we propose.

The cavity pattern could be correctly detected using video and automated readout of data was possible. Conventional methods use still images for accurate detection by visual inspection. However, it was possible to accurately detect cavity patterns even without such visual inspection. Generating and adding different images has contributed greatly to protect the copyrights of digital data for 3-D printers. It will be necessary in the future to extensively investigate conditions to establish more robust reading methods.

Fig. 4.
figure 4

Comparison between the method that process image in each period and our proposed techniques that process signal by summarized all image from video clip. First row, the original image at frame number (from left to right) 1, 50, 100, 150, 200, 300, respectively; Second row, the binarized image of cavities pattern signal at each frame processing and their percentage of recognition; Third row, the binarized image of summarized cavities pattern signal since the first to current image processing and their percentage of recognition; white squares means that a cavity is detected, and black squares means to no-cavities detection.

Full size image

5 Conclusion

Nondestructive readout by using thermography to reveal copyright information that was concealed in real objects and examined with the new method using video was proposed in this paper. Samples that had fine structures as cavities inside them were heated. After heating was stopped, thermographic videos were captured to measure the surface temperature distribution until samples were cooled to the original temperature. These videos were processed by image processing to find the cavity patterns inside the samples. First, a suitable image for processing for registration as a background image for these video clips was found during the period of heating until heating stopped. Then, different images between the registered background image and the images of individual frames after heating had stopped, up until samples had cooled to the original temperature, were calculated. Then, targets with only small differences in images were compared with surrounding areas to find the cavity pattern signal in the current image. The binary images that were obtained conformed to the rules that zeros meant no cavities and ones meant cavities. Finally, all the binary images were summarized to more clearly amplify the cavity pattern signal. Then, the final signal was generated to decode the embedded information formed in the samples. Such image processing made it possible to accurately detect cavities in the patterns. These results demonstrated the feasibility of automated readout of data with thermographic videos.