Lattice estimation from images of patterns that exhibit translational symmetry

doi:10.1016/j.imavis.2013.12.003

Image and Vision Computing

Volume 32, Issue 1, January 2014, Pages 64-73

https://doi.org/10.1016/j.imavis.2013.12.003 Get rights and content

Highlights

•
Regular texture patterns exhibiting translational symmetry were modelled.
•
Model comparison was implemented using Bayes Information Criterion.
•
Texel lattice geometry was estimated from e.g., images from a textile archive.
•
Accuracy was better than other methods with which it was compared.

Abstract

The analysis of regular texture images is cast in a model comparison framework. Texel lattice hypotheses are used to define statistical models which are compared in terms of their ability to explain the images. This approach is used to estimate lattice geometry from patterns that exhibit translational symmetry (regular textures). It is also used to determine whether images consist of such regular textures. A method based on this approach is described in which lattice hypotheses are generated using analysis of peaks in the image autocorrelation function, statistical models are based on Gaussian or Gaussian mixture clusters, and model comparison is performed using the marginal likelihood as approximated by the Bayes Information Criterion (BIC). Experiments on public domain images and a commercial textile image archive demonstrate substantially improved accuracy compared to several alternative methods.

Introduction

Regular textures can be modelled as consisting of repeated texture elements, or texels. The texels tesselate (tile) the image or, more generally, a surface. Varying illumination, varying physical characteristics of the textured surface, geometric deformations and sensor noise all result in images of such patterns exhibiting approximately regular, as opposed to exactly regular, texture. In this paper, we consider the task of automatically extracting texels from images. In particular we are interested in analysing patterns in printed textiles. Such patterns often exhibit translational symmetry and the visual structure within each texel can itself be complex. Translationally symmetric regular textures can always be generated by a pair of shortest vectors (two linearly independent directions), t₁ and t₂, that define the size, shape and orientation (but not the position) of the texel and the lattice that it generates. The lattice topology is always then quadrilateral. Such textures form one class of wallpaper pattern [1]. We restrict ourselves to consideration of images of planar, approximately regular textures viewed under orthographic projection. Whilst this might at first seem restrictive, this problem is, as will become apparent, far from completely solved. Furthermore, solutions will find application in analysis, retrieval and restoration of textile, wallpaper and tile design images, for example. The aim then, is to automatically decide whether a translationally symmetric regular texture class provides a good model for a texture image, and in cases in which it does, to extract the most predictive texel geometry.

In an earlier conference paper [2], this problem was cast in a statistical model comparison framework; models representing different hypotheses for lattice geometry were compared using the Bayesian Information Criterion (BIC). In this paper we extend this work in several ways. A more extensive discussion of related literature is provided setting the work more clearly in context, and certain aspects of the method have been clarified. False hypotheses are pruned using a constraint on the angle between t₁ and t₂. Classification of texture images as regular or irregular using model comparison is compared empirically to an alternative method [3], [4]. Model comparison is performed using the Akaike Information Criterion (AIC) as well as BIC. New experiments are reported on larger datasets, and results are reported separately for private and public datasets thus enabling future comparisons to be made using the public data. A detailed comparison of lattices obtained using several methods is presented. Experiments involving qualitative observer evaluations are expanded. The proposed method and implementations of two alternative methods from the literature are compared quantitatively against ground-truth annotations using two quantitative evaluation measures.

The rest of this paper is organized as follows. Section 2 outlines the most closely related previous work. Section 3 presents the model comparison framework. Section 4 describes details of lattice models. Section 5 describes the method used in our experiments for generating lattice hypotheses. Evaluations are described in Section 6 and conclusions are drawn in Section 7.

Section snippets

Related work

Previous work proposed for texel and lattice extraction can be grouped broadly into two categories: the local feature-based approach [5], [6], [7], [8], [9], [10], [11], [12], [13] and the global structure-based approach [1], [14], [15], [16], [17], [18]. All texture analysis is necessarily both local and global. The categorisation is in terms of the computational approach: whether it starts by identifying local features and proceeds to analyse global structure, or it starts with a global

Model comparison framework

The approach we take is to formulate texel hypotheses as statistical models and then compare these models given the image data. It is not sufficient for a model to be able to fit the data well. The best texel hypothesis under this criterion would be the image itself whereas our purpose is to extract the smallest texture element. Therefore, overfitting must be guarded against by penalising model complexity. Texel hypothesis comparison can be regarded as a model comparison problem for

Lattice models

Each of the Q clusters was modelled as Gaussian, a modelling choice motivated by exploratory data analysis which showed Gaussian-like distributions, as well as by parsimony and computational simplicity. Experimental results (see Section 6) suggest that this was a reasonable choice, at least for the data used here. However, when datasets contain many spatially localised disruptions (due to physical surface damage, local photometric variations such as specular highlights, local geometric

Lattice hypothesis generation

In principle there is an infinite number of lattice hypotheses. However, probability density will be highly concentrated at multiple peaks in the hypothesis space. The posterior distribution can therefore be well represented by only considering a, typically small, number of hypotheses at these peaks. In the maximum a posteriori setting adopted here, the approach taken is to identify multiple hypotheses in a data-driven manner and then compare these hypotheses using BIC. The approach is general

Evaluation

There are a few systematic evaluations of lattice extraction algorithms in the literature [29], [30], [19]. Chen et al. [29] reported an evaluation of translational symmetry in terms of the number of texels but not in terms of their shape. Park et al. [19] described an evaluation method for detection of deformed lattice regions in photographs of real-world scenes. Ground-truth lattice structures were annotated using an interactive editing tool. Detection rate was defined as the number of valid

Discussion and conclusions

It seems clear that the method proposed in this paper has superior accuracy to the methods with which it was compared. This was the case on both the commercial and public databases. In particular, the number of lattice extractions not judged to be clearly correct by observers was only a third of that obtained by the closest competing method. Even with parameter settings which disadvantage it relative to the other methods, the proposed method gave markedly better lattice estimates (Table 2).

Acknowledgements

The authors are grateful to Ruixuan Wang and Annette Ward for their helpful discussions. They would like to thank J. Hays and M. Park for providing their source code. This research was supported by the UK Technology Strategy Board grant “FABRIC: Fashion and Apparel Browsing for Inspirational Content” in collaboration with Liberty Art Fabrics, System Simulation Ltd., and the Victoria & Albert Museum.

References (33)

K.-L. Lee et al.
An efficient computation method for the texture browsing descriptor of MPEG-7
Image Vis. Comput.
(2005)
H. Lin et al.
Extracting periodicity of a regular texture based on autocorrelation functions
Pattern Recogn. Lett.
(1997)
D. Chetverikov
Pattern regularity as a visual key
Image Vis. Comput.
(2000)
J. Leu
On indexing the periodicity of image textures
Image Vis. Comput.
(2001)
F. Wu et al.
An evolutionary system for near-regular texture synthesis
Pattern Recogn.
(2007)
Y. Liu et al.
A computational model for periodic pattern perception based on frieze and wallpaper groups
IEEE Trans. Pattern Anal. Mach. Intell.
(2004)
J. Han et al.
Regular texture analysis as statistical model selection
K.L. Lee et al.
A new method for coarse classification of textures and class weight estimation for texture retrieval
Pattern Recogn. Image Anal.
(2002)
J. Hays et al.
Discovering texture regularity as a higher-order correspondence problem
W. Lin et al.
A lattice-based MRF model for dynamic near-regular texture tracking
IEEE Trans. Pattern Anal. Mach. Intell.
(2007)

T. Leung et al.

Detecting, localizing and grouping repeated scene elements from an image

T. Tuytelaars et al.

Noncombinational detection of regular repetitions under perspective skew

IEEE Trans. Pattern Anal. Mach. Intell.

(2003)

F. Schaffalitzky et al.

Geometric grouping of repeated elements within images

D.A. Forsyth

Shape from texture without boundaries

A. Lobay et al.

Recovering shape and irradiance maps from rich dense texton fields

M. Park et al.

Deformed lattice discovery via efficient mean-shift belief propagation

Cited by (8)

Fabric defect inspection based on lattice segmentation and template statistics
2020, Information Sciences
Citation Excerpt :
The former gets the information about the texture primitives before inferring the placement rule, while the latter infers the rule first and then obtains the texture primitives based on the rule. Generally, global feature-based methods are suitable for images containing large texture primitives of sophisticated appearances [13]. Consequently, global feature-based methods are commonly adopted in fabric inspection, e.g., lattice segmentation method in [16–18,23].
Automated fabric inspection is desirable for quality control of fabric industry. However, it is challenging due to some unpredictable fabric defects which may only occur during the production. Hence, methods aiming at automated fabric inspection are commonly developed in absence of defective samples. This paper proposes a novel automated fabric inspection method based on lattice segmentation and template statistics (LSTS) focusing on the patterned fabric images containing repetitive texture primitives. The proposed method attempts to infer the placement rule of texture primitives and divide the image into none-overlapping lattices as texture primitives which represent the given image by hundreds of lattices instead of millions of pixels. The defects are then efficiently identified by comparing the lattice similarity w. r. t. the benchmarks named template statistics. The template statistics are learnt from defect-free samples through a modular framework in which multiple feature extraction methods like Gabor filters and local binary pattern can be flexibly combined according to their inspection efficiencies. The performance of the proposed method is evaluated in the databases of dot-, box- and star-patterned fabric images. By comparing the resultant and ground-truth images, an overall detection rate of 0.977 is achieved, which is competitive with the state-of-the-arts.
Fabric defect inspection based on lattice segmentation and lattice templates
2018, Journal of the Franklin Institute
Citation Excerpt :
The former identifies the texture primitives before inferring the placement rule, while the latter reverses the order. Generally, GFM are suitable for images containing large texture primitives of sophisticated appearances [27]. Consequently, GFM are commonly adopted in AFI, e.g., Liu’s et al method in [28] and LS method in [17] and [18].
Automated fabric inspection is a challenging task due to the unpredictable visual forms of the fabric defects and their scarcity compared with the tremendous amount of defect-free fabric products. This paper proposes a novel method based on lattice segmentation and lattice templates which automatically identifies the defects of fabric images. With the proposed method, a fabric image is segmented to lattices by inferring the placement rule of the texture primitives categorized to distinct texture classes. Each texture class is modeled by multiple templates inferred from the defect-free samples based on some metrics determined a priori according to their inspection efficiencies. For a lattice segmented from a given image, the most similar template is identified through a template matching process which compensates the local deformations around the lattice, and the distances between the lattice and the identified template are estimated based on the selected metrics. The lattices of distances exceeding the learnt distance range are identified as defective. The performance of the proposed method is evaluated based on two databases respectively providing pixel-level and image-level evaluations. For both databases, the receiver operating characteristic curves are plotted and the average areas under curves are 0.86 and 0.95, respectively, for pixel-level and image-level databases. The proposed method is further tested on the blurred and noisy version of images from pixel-level database and the resulting area is 0.81 on average. The proposed method outperforms the state-of-the-art methods by comparing corresponding areas.
Fabric defect inspection based on isotropic lattice segmentation
2017, Journal of the Franklin Institute
Automated visual inspection of fabric defects is a challenge due to the diversity of the fabric patterns and defects. Although there are many automated inspection methods of identifying fabric defects, most methods process images containing the fabric patterns classified as the crystallographic group p1 and implicitly assume the fabric patterns are arranged in fixed directions. This paper proposes an automated defect inspection method which calibrates the fabric image and then segments the image into none-overlapped sub-images which are called lattices. Thus, the image is represented by hundreds of lattices sharing some common features instead of millions of unrelated pixels. The defect inspection problem is transformed to comparing the lattice similarity based on the shared features and identifying the defective lattices as the outliers in the feature space. The performance of the proposed method ILS (Isotropic Lattice Segmentation) is evaluated on the databases of images containing fabric patterns arranged orthogonally and arbitrarily. By comparing the resultant images with ground-truth images, an overall detection rate of 0.955 is achieved, which is comparable with the state-of-the-art methods.
Fabric defect inspection based on lattice segmentation and Gabor filtering
2017, Neurocomputing
Citation Excerpt :
The following literature review shortly discusses each component related with fabric defect inspection. The lattice segmentation may be categorized based on their computational approaches [25], i.e., the local feature-based and the global structure-based approaches. The former identifies the local features first and then analyzes the global structure, the latter conducts the two steps in a reverse order.
Fabric defect inspection aims at detecting the defects presented on a patterned fabric surface to achieve high quality. However, visual inspection is challenging due to the diversity of the fabric patterns and defects. This paper presents an automatic defect inspection method which compares the similarities of semantic sub-images conformed to crystallographic groups called lattice. The lattices are automatically segmented based on morphological component analysis (MCA). The defect inspection is then formulated as a novel voting procedure depending on an ideal lattice artificially generated by investigating the distributions of responses given by convolving lattices with Gabor filters. The performance of the proposed method LSG (lattice segmentation assisted by Gabor filters) is evaluated on the databases of star- and box-pattern images. By comparing the resultant and ground-truth images, an overall detection rate of 0.975 is achieved, which is comparable with the state-of-the-art methods.
Fabric defect fetection via weighted low-rank decomposition and Laplacian regularization
2020, Journal of Engineered Fibers and Fabrics
Analysis of Planar Ornament Patterns via Motif Asymmetry Assumption and Local Connections
2019, Journal of Mathematical Imaging and Vision

View all citing articles on Scopus

^☆: This paper has been recommended for acceptance by Enrique Dunn, Ph.D.

^☆☆: This research was carried out whilst J. Han was with the School of Computing, University of Dundee, DD1 4HN, UK.

View full text

Lattice estimation from images of patterns that exhibit translational symmetry☆,☆☆

Highlights

Abstract

Introduction

Section snippets

Related work

Model comparison framework

Lattice models

Lattice hypothesis generation

Evaluation

Discussion and conclusions

Acknowledgements

Image Vis. Comput.

Pattern Recogn. Lett.

Image Vis. Comput.

Image Vis. Comput.

Pattern Recogn.

A computational model for periodic pattern perception based on frieze and wallpaper groups

IEEE Trans. Pattern Anal. Mach. Intell.

Regular texture analysis as statistical model selection

A new method for coarse classification of textures and class weight estimation for texture retrieval

Pattern Recogn. Image Anal.

Discovering texture regularity as a higher-order correspondence problem

A lattice-based MRF model for dynamic near-regular texture tracking

IEEE Trans. Pattern Anal. Mach. Intell.

Detecting, localizing and grouping repeated scene elements from an image

Noncombinational detection of regular repetitions under perspective skew

IEEE Trans. Pattern Anal. Mach. Intell.

Geometric grouping of repeated elements within images

Shape from texture without boundaries

Recovering shape and irradiance maps from rich dense texton fields

Deformed lattice discovery via efficient mean-shift belief propagation