Fabric defect inspection based on lattice segmentation and lattice templates

https://doi.org/10.1016/j.jfranklin.2018.07.005Get rights and content

Abstract

Automated fabric inspection is a challenging task due to the unpredictable visual forms of the fabric defects and their scarcity compared with the tremendous amount of defect-free fabric products. This paper proposes a novel method based on lattice segmentation and lattice templates which automatically identifies the defects of fabric images. With the proposed method, a fabric image is segmented to lattices by inferring the placement rule of the texture primitives categorized to distinct texture classes. Each texture class is modeled by multiple templates inferred from the defect-free samples based on some metrics determined a priori according to their inspection efficiencies. For a lattice segmented from a given image, the most similar template is identified through a template matching process which compensates the local deformations around the lattice, and the distances between the lattice and the identified template are estimated based on the selected metrics. The lattices of distances exceeding the learnt distance range are identified as defective. The performance of the proposed method is evaluated based on two databases respectively providing pixel-level and image-level evaluations. For both databases, the receiver operating characteristic curves are plotted and the average areas under curves are 0.86 and 0.95, respectively, for pixel-level and image-level databases. The proposed method is further tested on the blurred and noisy version of images from pixel-level database and the resulting area is 0.81 on average. The proposed method outperforms the state-of-the-art methods by comparing corresponding areas.

Introduction

As an industrial product possessing the most diverse two-dimensional surfaces, fabric (textile) serves many fields of human civilization and is inseparable from our daily lives. The number of fabric products is tremendous and the quality control thus plays an important role in saving cost [1]. A critical aspect of quality control is inspecting fabric defects of unpredictable visual forms which randomly occur in the automatic manufacturing process. Consequently, it is difficult to collect lots of defective fabric samples. Hence, Automated Fabric Inspection (AFI) which identifies defects of unpredictable visual forms is always developed in absence of defective samples, which makes AFI as a challenging task. As a result, there are numerous AFI methods developed for various fabrics. These methods may be categorized from different perspectives such as the texture representations and the fabric types.

According to the texture representations, AFI methods may be categorized to four classes [2], [3], [4], i.e., statistical [5], [6], [7], spectral [8], [9], [10], model-based methods [11], [12], [13] and Structural Analysis (SA) [14], [15], [16]. Statistical methods based on the gray values are found not good at identifying subtle defects [8]. Spectral methods mainly focus on the patterned fabric images consisting of the repeated texture primitives whose periodicity can be revealed more easily in frequency domain than in spatial domain. Therefore, spectral methods commonly require the fabric should be explicitly periodic [4]. Model-based methods represent texture characteristics as the parameters of some statistical models like autoregressive models and Markov Random Field (MRF). It is found that small defects cannot be efficiently identified by using model-based methods [8]. SA methods represent the fabric texture by texture primitives (texels) [3], [4] repeated according to some placement rules which can be random [16] or specific [14], [15]. The texture primitives are modeled differently in SA methods. For instance, texture primitive is defined as the runs of the foreground pixels in the binarized fabric image in [14], and the defects are identified based on the analyses of the histograms associated with the run locations and lengths. Run locations serve as the placement rule to assist the defect inspection. In [15], texture primitive is defined as texture blobs enclosed by the rectangular regions within a grid overlapping the binarized image. The defects are identified by comparing the maximum frequency differences of the texture primitives. Unlike [14], the inference of the placement rule is omitted in [15] because texture primitives are assumed to be arranged in a rigid grid. Contrary to the rigid grid adopted globally for all images as reported in [15], an inference process revealing a dynamic grid for a single given image is adopted to segment texture primitives in both [17], [18]. The inference is derived based on the locations of texture blobs and thus the deformation of texture primitives are partially taken into account. Therefore, the dynamic grid changes correspondingly to conform the texture blob positions which differ from image to image. The rectangular region determined by the dynamic grid is named lattice in [17], [18]. The advantage of lattice-based methods is that a given image is represented by hundreds of lattices instead of millions of pixels, which makes computationally expansive operations practical in AFI.

On the basis of fabric types, AFI methods may be categorized to two classes [3], i.e., methods capable or incapable of processing fabrics categorized as groups other than p1 group of wallpaper group [19]. Wallpaper groups categorize two dimensional repetitive patterns according to their symmetries. The symmetry defines a set of rules to generate the pattern based on some smallest texture called motif. Essentially, there are four basic rules to build symmetry, i.e., translations; rotations, reflections and glide reflections. The group p1 contains only translations; there are no rotations, reflections, or glide reflections. Most of the aforementioned methods except [17], [18] aim at the plain or twill fabric categorized as p1 group [19]. A few are able to process fabrics of groups other than p1 group [3], e.g., Wavelet-pre-processed Golden Image Subtraction (WGIS) [20], Bollinger Bands (BB) [21], Regular Bands (RB) [22], Image Decomposition (ID) [23], Motif-Based (MB) method [19] and Elo Rating (ER) method [24]. BB is developed based on the observation that defects break the regularity of runs of the pixel values along rows and columns in fabric image. The regularities of the horizontal and vertical runs of pixels are measured by computing the values named upper and lower bands. The defects are identified as the pixels whose bands exceed the band ranges learnt from defect-free samples only. RB inherits the spirit of BB, and its framework is very similar to that of BB except that the regularity is evaluated by computing the light and dark regular bands instead of the lower and upper bands. Besides the regularity of pixel runs, templates are another popular means to inspect defects. WGIS works based on the subtraction between a manually selected template and pixels enclosed in a window. The subtraction is conducted according to the level-1 approximation yielded by Haar wavelet for reducing the noises. The defects are identified as the pixels whose subtraction exceeds a range learnt from the defect-free samples only. Another template-based method named ER [24] is similar to WGIS except that the subtraction in [20] is replaced by a matching process inspired by the Elo rating system for evaluating the chess players’ capability. Essentially, a patch of predefined size is compared with several randomly chosen patches for a given fabric image by computing scores involving their subtractions. The defects are identified as the pixels which are centers of the patches with scores exceeding the range learnt from the defect-free samples only.

This paper proposes a method based on lattice segmentation and modular metric framework capable of processing images of fabrics from groups other than p1. The fabric image is assumed to consist of patterns repeated in a manner consistent horizontally and vertically in the image, the patterns lead to texture blobs in the binary version of the image, and the texture blobs are aligned with the image axes. For a given fabric image meeting this assumption, lattice segmentation infers a grid overlapping on the image to capture the patterns in a fine texture granularity. Especially for the alternatively changed patterns, textures enclosed by the lattices also change periodically. For each type of texture during a period, it is modeled by a group of templates generated based on lattices of this type. The modeling process incorporates distance metrics for measuring lattice texture similarities, and the metrics serve as replaceable module in the process. Thus, the proposed method can be flexibly adjusted according to its efficiency of the application on hand. Generally, the contributions of this paper are summarized as follows.

  • (1)

    A novel fabric inspection algorithm based on Lattice Segmentation and Lattice Templates (LSLT) is proposed. Contrary to the traditional methods involving some feature extractions of the fixed structures, the proposed algorithm can flexibly combine the distance metrics found efficient for the current application, which leads to the great adaptability.

  • (2)

    The proposed algorithm is tested on multiple databases including one containing the blurred and noisy fabric images rarely found in fabric inspection literatures. The experiment results are represented as ROC curves which are rarely adopted but effectively reflect the performances of AFI methods throughout fabric inspection literatures.

The following parts of this paper are organized as follows. In Section 2, the reported algorithms involved in the proposed method are briefly introduced. In Section 3, the novel lattice segmentation and the modular framework of the proposed method are outlined. In Section 4, the optimal metric combination and the corresponding performance are evaluated. Lastly, Section 5 is the conclusion of the paper.

Section snippets

Related works

The basics of the proposed method involve several reported algorithms which can be categorized to two classes based on their functionalities. The two classes are image decomposition, and lattice segmentation.

Image decomposition differs from the common image enhance techniques which indistinguishably enhance the edges and textures, and it generates an edge-enhanced version called carton or structural image in which only edges are enhanced and the rest are blurred. Thus, Image decomposition may

The LSLT method

Suppose there are I1,I2,,IN defect-free fabric images consisting of orthogonally repeated texture primitives categorized to t distinct classes, the classes may be modeled by some templates reflecting the most common characteristics shared by the texture primitives. However, there are three issues about this modeling process: (1) how the fabric image can be segmented to lattices representing the texture primitives of different classes; (2) there may be pathological data in some training

Performance evaluation

The performance evaluation of the proposed method (LSLT) essentially involves two fabric image databases providing pixel- and image-level evaluation, respectively. Pixel-level evaluation is conducted based on Fabric Image Database (FID) provided by Industrial Automation Research Laboratory from Dept. of Electrical and Electronic Engineering of Hong Kong University. Image-level evaluation is implemented on Fabric Defect Detection Database (FDDD) [4], [31], [32] provided by Department of Computer

Conclusion

In this paper, a novel AFI method is proposed to identify fabric defects based on the defect-free fabric images. Because the defective samples are assumed unavailable, LSLT method is derived purely based on the defect-free samples. For a given fabric image, with the LSLT method, the image is segmented to non-overlapping lattices through a novel lattice segmentation procedure and then the lattices are compared with the lattice templates inferred from defect-free samples. The lattice segmentation

Acknowledgments

This research is financially supported by the National Natural Science Foundation of China (Grant No. 31670553).

References (33)

  • SongK.Y. et al.

    Defect detection in random colour textures

    Image Vis. Comput.

    (1996)
  • JiaL. et al.

    Fabric defect inspection based on lattice segmentation and Gabor filtering

    Neurocomputing

    (2017)
  • JiaL. et al.

    Fabric defect inspection based on isotropic lattice segmentation

    J. Frankl. Inst.

    (2017)
  • NganH.Y.T. et al.

    Motif-based defect detection for patterned fabric

    Pattern Recognit.

    (2008)
  • NganH.Y.T. et al.

    Wavelet based methods on patterned fabric defect detection

    Pattern Recognit.

    (2005)
  • TsangC.S.C. et al.

    Fabric inspection based on the Elo rating method

    Pattern Recognit.

    (2016)
  • Cited by (16)

    • Fabric defect inspection based on lattice segmentation and template statistics

      2020, Information Sciences
      Citation Excerpt :

      Namely, the texture primitives enclosed by the lattices can be categorized to t different types. To recover the lattice period of a fabric image, a period estimation based on Fourier transformation reported in [18] is employed. The estimation process is explained as follows.

    • Exploratory Data Analysis for Textile Defect Detection

      2023, International Journal on Recent and Innovation Trends in Computing and Communication
    View all citing articles on Scopus
    View full text