Geometric Object 3D Reconstruction from Single Line Drawing Image Based on a Network for Classification and Sketch Extraction

Abstract

Geometric objects in educational materials are usually illustrated as 2D line drawings without depth information, and this is a barrier for readers to fully understand the 3D structure of these geometric objects. To address this issue, we propose a novel method to recover the 3D shape of the geometric object from a single 2D line drawing image. Specifically, our proposed method can be divided into two stages: sketch extraction stage and reconstruction stage. In the sketch extraction stage, we propose a deep neural network to identify the category of the geometric object in the line drawing image and extract its sketch simultaneously. Our network architecture is based on High-Resolution Network (HRNet), which integrates two task-specific decoders: one for classification and the other for vertices detection. With the predicted category and location of vertices, we can easily obtain the sketch of the geometric object in the input line drawing image. Compared with previous methods, our CNN-based method can directly extract the sketch of geometric objects without any hand-crafted features or processes, which gives a more robust performance. In the reconstruction stage, we exploit an example-based method and conduct the reconstruction by optimizing an objective function of reconstruction error. Moreover, we generate a simulated dataset to alleviate the problem caused by unbalanced distribution across different categories in the manually-collected dataset, which greatly improves the performance of our deep neural network model. Extensive experimental results demonstrate that the proposed method performs significantly better than the existing methods in both accuracy and efficiency.