On visual similarity based 2D drawing retrieval

doi:10.1016/j.cad.2005.10.009

Computer-Aided Design

Volume 38, Issue 3, March 2006, Pages 249-259

https://doi.org/10.1016/j.cad.2005.10.009 Get rights and content

Abstract

A large amount of 2D drawings have been produced in engineering fields. To reuse and share the available drawings efficiently, we propose two methods in this paper, namely 2.5D spherical harmonics transformation and 2D shape histogram, to retrieve 2D drawings by measuring their shape similarity. The first approach represents a drawing as a spherical function by transforming it from a 2D space into a 3D space. Then a fast spherical harmonics transformation is employed to get a rotation invariant descriptor. The second statistics-based approach represents the shape of a 2D drawing using a distance distribution between two randomly sampled points. To allow users to interactively emphasize certain local shapes that they are interested in, we have adopted a flexible sampling strategy by specifying a bias sampling density upon these local shapes. The two proposed methods have many valuable properties, including transform invariance, efficiency, and robustness. In addition, their insensitivity to noise allows for the user's causal input, thus supporting a freehand sketch-based retrieval user interface. Experiments show that a better performance can be achieved by combining them together using weights.

Introduction

As a principal way to express and communicate design ideas, a large number of 2D drawings in engineering fields (e.g., architecture and industrial domains) have been produced in the past decades. Reusing and sharing these drawing efficiently is becoming an important way to accelerate the design process, improve product quality, and reduce costs. However, their proliferation makes it difficult for engineers to retrieve the desired drawings. Traditionally, there have been several ways, such as keywords, encoding approach and tree-like structure navigation, for users to find a drawing. Although these approaches are simple to implement, they are always time-consuming and not sufficient to describe a 2D drawing precisely since the design ideas are represented explicitly in its geometric shape. Therefore, it is necessary to provide users with a way to retrieve a 2D drawing from the shape perspective.

As a fundamental research topic in computer vision and robotics, 2D shape recognition has been studied thoroughly, and many methods have also been proposed. Most of these methods concentrate on the contour matching between objects because there is a popular belief that it is the outline of an object that leads to the concept of shape [1], [2], [3]. However, it is hard to apply these contour-based methods to a 2D drawing since drawings usually have complex internal structures. For example, Fig. 1(a) shows a vector drawing composed of a series of lines, arcs, and circles, while Fig. 1(b) shows its contour. It can be seen that they represent two different shape concepts from an engineering perspective. If the internal structures of a drawing are ignored, the most important information will be missed. It is worth noting that there is difference between drawing understanding and shape recognition. Drawing understanding tries to recognize the graphic entities (such as lines, arcs, and circles) and specific engineering concepts (such as dimension and text) contained in the scanned drawings, and its purpose is to convert engineering drawings in paper or sketched from into a CAD form. Shape recognition only focuses on the geometric shape and its aim is to interpret what kind of object the shape represents or compute the similarity between two arbitrary shapes.

2D drawing retrieval is related to 2D shape recognition. As shown in the block diagram in Fig. 2, we can define 2D drawing retrieval as: given a drawing A and a drawing library L={B_i|0≤i≤n}, how to compute the similarity distance A and B_i, i.e. D(A, B_i), and find the k-nearest drawings within a certain tolerance ɛ. In the retrieval process, the determination of a proper shape descriptor is the key to 2D drawing retrieval. In this paper, we propose two methods to compute the shape similarity between 2D drawings. The first approach represents a drawing as a spherical function by transforming it from 2D space into 3D space. Then we employ a fast spherical harmonics transformation on the 2.5D object to get a rotation invariant descriptor. The second method represents the shape of a 2D drawing from the statistics perspective as a distance distribution between two randomly sampled points.

The remainder of this paper is organized as follows. In Section 2, some related work is introduced. Then the two proposed methods are described in 3 Spherical harmonics representation, 4 2D Shape histogram, respectively. To evaluate the performance of these methods and their combination, some experiments are presented in Section 5. In Section 6, the conclusion and future work are given.

Section snippets

Related work

As a compact way to describe the shape of an object, 2D contour is widely used to recognize an object. Well-known methods for 2D contour matching include Fourier descriptors [4], curvature scale space (CSS) [5], chain codes [6], Zernike moments [7], and the classical Hausdorff distance [8]. By representing 2D shape as a series of points sampled along the shape boundary, McConnell et al. [1] used the turning of a tangent formed by consequent points sampled on the contours of objects to measure

Spherical harmonics transform

Spherical harmonics representation has been successfully applied to 3D shape matching [23] as a robust rotation invariant descriptor. It arises on the sphere in the same way that the Fourier exponential function arises on the circle. According to the theory of spherical harmonics, a function f(θ, φ) represented in a spherical coordinate can be approximated with a sum of its spherical harmonics $Y_{l}^{m} (θ, φ)$ : $f (θ, φ) = \sum_{l = 0}^{\infty} \sum_{m = - l}^{m = l} a_{l, m} Y_{l}^{m} (θ, φ)$ where {a_l,m} are the coefficients in the frequency domain, $Y_{l}^{m}$

2D Shape histogram

To measure the similarity between 3D shapes, Osada et al. [25] represented a 3D shape as a shape distribution signature that is formed by random points sampled uniformly from the shape surface. This method has many valuable advantages: invariant to affine transformation, insensitive to noise or cracks, simple, and fast. Under the inspiration of this method, we derive a 2D shape distribution analog. Experiments in Section 5 show this derivation is good at computing the similarity between 2D

Experiments and discussion

The two methods introduced in the preceding sections have been incorporated into a 2D and 3D shape retrieval system called ShapeLab. In order to test the performance of the two methods, we have built a benchmark, namely Purdue 2D and 3D Shape Benchmark, which includes 2000 2D drawings from industrial fields. These drawings are classified into 50 clusters from simple to complex according to their functions and geometric shape. We will introduce our implemented retrieval system, i.e. ShapeLab,

Conclusion

In this paper, we have proposed two methods to compute the similarities between 2D drawings. As two different rotation invariant descriptors, both these methods can provide a compact representation of a 2D drawing. The experiments show that they are efficient and have good discriminative ability and can be applied to vector drawings and scanned drawings. Since the two proposed methods are not insensitive to noise and the similarity measurements are conducted in 2D space, they also support

Jiantao Pu is a postdoctoral scholar at PRECISE (the Purdue Research and Education Center for Information System in Engineering), Purdue University. His research interests lie in Computer Graphics (real-time rendering, geometric processing, and animation), Computer Vision, Virtual Reality, and Human Computer Interaction. He has published almost thirty papers and filed three patents.

References (26)

J. Park et al.
A new approach to similarity retrieval of 2D graphic objects based on dominant shapes
Pattern Recogn Lett
(1999)
R. McConnell et al.
Ψ-S correlation and dynamic time warping: two methods for tracking ice floes in SAR images
IEEE Trans Geosci Remote Sens
(1991)
S. Gold et al.
A graduated assignment algorithm for graph matching
IEEE Trans Pattern Anal Mach Intell
(1996)
S. Belongie et al.
Shape matching and object recognition using shape context
IEEE Trans Pattern Anal Mach Intell
(2002)
H. Kauppinen et al.
An experimental comparison of autoregressive and fourier-based descriptors in 2D shape classification
IEEE Trans Pattern Anal Mach Intell
(1995)
Mokhtarian F, Abbasi S, Kittler J. Robust and efficient shape indexing through curvature scale space. In: Proceedings...
H. Freeman
Computer processing of line-drawing images
Comput Surv
(1974)
A. Khotanzad et al.
Invariant image recognition by zernike moments
IEEE Trans Pattern Anal Mach Intell
(1990)
Alt H, Behrends B, Blömer J. Approximate matching of polygonal shapes. Proceedings of the seventh annual symposium on...
Sebastian TB, Klein PH, Kimia BB. Alignment-based recognition of shape outlines, Proceedings of the fourth...

Jeannin S. Mpeg-7 visual prt of eXperimentation model version 9.0. In: ISO/IEC JTC1/SC29/WG11/N3914, 55th MPEG meeting,...

Liu T, Geiger D. Approximate tree matching and shape similarity. Proceedings of the seventh IEEE international...

Klein PH, Tirthapura S, Sharvit D, Kimia BB. A tree-edit distance algorithm for comparing simple closed shapes. In:...

Cited by (93)

‘CADSketchNet’ - An Annotated Sketch dataset for 3D CAD Model Retrieval with Deep Neural Networks
2021, Computers and Graphics (Pergamon)
Ongoing advancements in the fields of 3D modelling and digital archiving have led to an outburst in the amount of data stored digitally. Consequently, several retrieval systems have been developed depending on the type of data stored in these databases. However, unlike text data or images, performing a search for 3D models is non-trivial. Among 3D models, retrieving 3D Engineering/CAD models or mechanical components is even more challenging due to the presence of holes, volumetric features, presence of sharp edges etc., which make CAD a domain unto itself. The research work presented in this paper aims at developing a dataset suitable for building a retrieval system for 3D CAD models based on deep learning. 3D CAD models from the available CAD databases are collected, and a dataset of computer-generated sketch data, termed ‘CADSketchNet’, has been prepared. Additionally, hand-drawn sketches of the components are also added to CADSketchNet. Using the sketch images from this dataset, the paper also aims at evaluating the performance of various retrieval system or a search engine for 3D CAD models that accepts a sketch image as the input query. Many experimental models are constructed and tested on CADSketchNet. These experiments, along with the model architecture, choice of similarity metrics are reported along with the search results.
A learning framework for shape retrieval based on multilayer perceptrons
2019, Pattern Recognition Letters
With the rapid development of 3D technology, the demand to use and retrieve 3D models has become increasingly urgent. In this paper, we present a framework that consists of a sketch-based local binary pattern (SBLBP) feature extraction method, a learning algorithm for the best view of a shape based on multilayer perceptrons (MLPs) and a learning method for shape retrieval based on two Siamese MLP networks. The model is first projected into many multiview images. A transfer learning scheme based on graphic traversal to identify Harris key points is proposed to build relations between view images and sketches. In addition, an MLP classifier is used for classification to obtain the best views of each model. Moreover, we propose a new learning method for shape retrieval that simultaneously uses two Siamese MLP networks to learn SBLBP features. Furthermore, we build a joint Bayesian method to fuse the outputs of the views and sketches. Based on training with many samples, the MLP parameters are effectively fit to perform shape retrieval. Finally, an experiment is conducted to verify the feasibility of the approach, and the results show that the proposed framework is superior to other approaches.
Clustering of architectural floor plans: A comparison of shape representations
2017, Automation in Construction
Generative design methods are able to produce a large number of potential solutions of architectural floor plans, which may be overwhelming for the decision-maker to cope with. Therefore, it is important to develop tools which organise the generated data in a meaningful manner. In this study, a comparative analysis of four architectural shape representations for the task of unsupervised clustering is presented. Three of the four shape representations are the Point Distance, Turning Function, and Grid-Based model approaches, which are based on known descriptors. The fourth proposed representation, Tangent Distance, calculates the distances of the contour's tangents to the shape's geometric centre. A hierarchical agglomerative clustering algorithm is used to cluster a synthetic dataset of 72 floor plans. When compared to a reference clustering, despite good perceptual results with the use of the Point Distance and Turning Function representations, the Tangent Distance descriptor (Rand index of 0.873) provides the best results. The Grid-Based descriptor presents the worst results.
Direct method for rotation estimation from spherical images using 3D mesh surfaces with SPHARM representation
2016, Journal of Visual Communication and Image Representation
Since a large field of view obviously bears important advantages, the use of spherical images is becoming increasingly important in various computer vision and image processing applications. This paper presents a novel rotation estimation approach for spherical images based on 3D mesh representation of gray level intensity. Once the 3D meshes of the underlying spherical images are obtained, the 3D rotation can be estimated directly and efficiently, without feature extraction and matching process. Subsequently, we propose a direct method for 3D object rotation estimation using spherical harmonics representation with SVD decomposition and ICP algorithm for estimation refinement. Experimental results validate our approach and prove its suitability and robustness for rotation estimation. Moreover, it performs well against noisy images, brightness changes, image compression and occlusions. A comparative study of our proposed approach with four similar methods for 3D rotation estimation between spherical images, is realized to prove its effectiveness.
Individual strategies in the tasks of graphical retrieval of technical drawings
2015, Journal of Visual Languages and Computing
A designer׳s information need can have a lot of finer points not expressible completely by textual attributes or global aggregated image features. The purpose of the study is to develop a cognitive environment for content-based engineering drawing retrieval, allowing a user to concretize queries, to implement strategies that are most effective for the current search task. We propose meaningful customizable graphic search patterns, various comparison modes and visualization of matched elements. The presented experiments show that the approach yields good results in the sense of relevance and is comfortable in terms of controllability. The possibility to customize search patterns can result in improving retrieval precision by 10–15%. In practice, it can significantly increase the degree of automation of the design-reuse process.
A new sketch-based 3D model retrieval approach by using global and local features
2014, Graphical Models
With the rapid growth of available 3D models, fast retrieval of suitable 3D models has become a crucial task for industrial applications. This paper proposes a novel sketch-based 3D model retrieval approach which utilizes both global feature-based and local feature-based techniques. Unlike current approaches which use either global or local features, as well as do not take into account semantic relations between local features, we extract these two kinds of feature information from the representative 2D views of 3D models that can facilitate semantic description and retrieval for 3D models. Global features represent the gross exterior boundary shape information, and local features describe the interior details by compact visual words. Specifically, an improved bag-of-features method is provided to extract local features and their latent semantic relations. In addition, an efficient two-stage matching strategy is used to measure the distance between the query sketch and 3D models for selection and refinement. Experiment results demonstrate that our approach which combines these two kinds of complementary features significantly outperforms several state-of-the-art approaches.

View all citing articles on Scopus

Karthik Ramani is a Professor in the School of Mechanical Engineering at Purdue University. He earned his B.Tech from the Indian Institute of Technology, Madras, in 1985, an MS from The Ohio State University, in 1987, and a Ph.D. from Stanford University in 1991, all in Mechanical Engineering. He has worked as a summer intern in Delco Products, Advanced Composites, and as a summer faculty intern in Dow Plastics, Advanced Materials. He was awarded the Dupont Young Faculty Award, the National Science Foundation Research Initiation Award, the National Science Foundation CAREER Award, the Ralph Teetor Educational Award from the Society of Automotive Engineers, Outstanding Young Manufacturing Engineer Award from the Society of Manufacturing Engineers, and the Ruth and Joel Spira Award for Outstanding contributions to the Mechanical Engineering Curriculum. In 2002, he was recognized by Purdue University through a University Faculty Scholars Award. In 2005 he won the Discovery in Mechanical Engineering Award for his work in shape search. He has developed many successful new courses—Computer-Aided Design and Prototyping, Product and Process Design and co-developed an Intellectual Property course. He founded the Purdue Research and Education Center for Information Systems in Engineering (PRECISE) and ToolingNET, a collaborative 21st century project funded by the State of Indiana. A major area of emphasis in his group is shape representations for search and configuration in both engineering and biology. His research is funded by the NSF, DLA/Army, and the National Institute of Health. He also chairs an ASME Computers and Information in Engineering Committee and is on the editorial board of Computer-Aided Design.

View full text

On visual similarity based 2D drawing retrieval

Abstract

Introduction

Section snippets

Related work

Spherical harmonics transform

2D Shape histogram

Experiments and discussion

Conclusion

Pattern Recogn Lett

Ψ-S correlation and dynamic time warping: two methods for tracking ice floes in SAR images

IEEE Trans Geosci Remote Sens

A graduated assignment algorithm for graph matching

IEEE Trans Pattern Anal Mach Intell

Shape matching and object recognition using shape context

IEEE Trans Pattern Anal Mach Intell

An experimental comparison of autoregressive and fourier-based descriptors in 2D shape classification

IEEE Trans Pattern Anal Mach Intell

Computer processing of line-drawing images

Comput Surv

Invariant image recognition by zernike moments

IEEE Trans Pattern Anal Mach Intell