Chufeng Xiao
Hongbo Fu
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Multiple sketch datasets have been proposed to understand how people draw 3D objects. However, such datasets are often of small scale and cover a small set of objects or categories. In addition, these datasets contain freehand sketches mostly from expert users, making it difficult to compare the drawings by expert and novice users, while such comparisons are critical in informing more effective sketch-based interfaces for either user groups. These observations motivate us to analyze how differently people with and without adequate drawing skills sketch 3D objects. We invited 70 novice users and 38 expert users to sketch 136 3D objects, which were presented as 362 images rendered from multiple views. This leads to a new dataset of 3,620 freehand multi-view sketches, which are registered with their corresponding 3D objects under certain views. Our dataset is an order of magnitude larger than the existing datasets. We analyze the collected data at three levels, i.e., sketch-level, stroke-level, and pixel-level, under both spatial and temporal characteristics, and within and across groups of creators. We found that the drawings by professionals and novices show significant differences at stroke-level, both intrinsically and extrinsically. We demonstrate the usefulness of our dataset in two applications: (i) freehand-style sketch synthesis, and (ii) posing it as a potential benchmark for sketch-based 3D reconstruction. Our dataset and code are available at https://chufengxiao.github.io/DifferSketching/.
Supplemental Material
- Itamar Berger, Ariel Shamir, Moshe Mahler, Elizabeth Carter, and Jessica Hodgins. 2013. Style and abstraction in portrait sketching. ACM Transactions on Graphics (TOG) 32, 4 (2013), 1--12.Google Scholar
Digital Library
- Gedas Bertasius, Jianbo Shi, and Lorenzo Torresani. 2015. Deepedge: A multi-scale bifurcated deep network for top-down contour detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4380--4389.Google ScholarCross Ref
- Ayan Kumar Bhunia, Ayan Das, Umar Riaz Muhammad, Yongxin Yang, Timothy M Hospedales, Tao Xiang, Yulia Gryaditskaya, and Yi-Zhe Song. 2020. Pixelor: A Competitive Sketching AI Agent. So you think you can sketch? ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--15.Google ScholarDigital Library
- Lucia Calabrese and Francesco S Marucci. 2006. The influence of expertise level on the visuo-spatial ability: differences between experts and novices in imagery and drawing abilities. Cognitive Processing 7, 1 (2006), 118--120.Google ScholarCross Ref
- John Canny. 1986. A computational approach to edge detection. IEEE Transactions on pattern analysis and machine intelligence 6 (1986), 679--698.Google ScholarDigital Library
- Shu-Yu Chen, Wanchao Su, Lin Gao, Shihong Xia, and Hongbo Fu. 2020. DeepFace-Drawing: Deep Generation of Face Images from Sketches. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2020) 39, 4 (2020), 72:1--72:16.Google Scholar
- Christopher B Choy, Danfei Xu, JunYoung Gwak, Kevin Chen, and Silvio Savarese. 2016. 3d-r2n2: A unified approach for single and multi-view 3d object reconstruction. In European conference on computer vision. Springer, 628--644.Google ScholarCross Ref
- Dale J Cohen and Susan Bennett. 1997. Why can't most people draw what they see? Journal of Experimental Psychology: Human Perception and Performance 23, 3 (1997), 609.Google ScholarCross Ref
- Forrester Cole, Aleksey Golovinskiy, Alex Limpaecher, Heather Stoddart Barros, Adam Finkelstein, Thomas Funkhouser, and Szymon Rusinkiewicz. 2008. Where do people draw lines? In ACM SIGGRAPH 2008 papers. 1--11.Google ScholarDigital Library
- Doug DeCarlo, Adam Finkelstein, Szymon Rusinkiewicz, and Anthony Santella. 2003. Suggestive contours for conveying shape. In ACM SIGGRAPH 2003 Papers. 848--855.Google ScholarDigital Library
- Johanna Delanoy, Mathieu Aubry, Phillip Isola, Alexei A Efros, and Adrien Bousseau. 2018. 3d sketching using multi-view deep volumetric prediction. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 1 (2018), 1--22.Google ScholarDigital Library
- Daniel Dixon, Manoj Prasad, and Tracy Hammond. 2010. icandraw: Using sketch recognition and corrective feedback to assist a user in drawing human faces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 897--906.Google ScholarDigital Library
- Dong Du, Xiaoguang Han, Hongbo Fu, Feiyang Wu, Yizhou Yu, Shuguang Cui, and Ligang Liu. 2020. SAniHead: Sketching Animal-like 3D Character Heads Using a View-surface Collaborative Mesh Generative Network. IEEE Transactions on Visualization and Computer Graphics (2020).Google Scholar
- Mathias Eitz, James Hays, and Marc Alexa. 2012a. How do humans sketch objects? ACM Transactionson graphics (TOG) 31, 4 (2012), 1--10.Google Scholar
- Mathias Eitz, Ronald Richter, Tamy Boubekeur, Kristian Hildebrand, and Marc Alexa. 2012b. Sketch-based shape retrieval. ACM Transactions on graphics (TOG) 31, 4 (2012), 1--10.Google Scholar
- Haoqiang Fan, Hao Su, and Leonidas J Guibas. 2017. A point set generation network for 3d object reconstruction from a single image. In Proceedings of the IEEE conference on computer vision and pattern recognition. 605--613.Google ScholarCross Ref
- Hongbo Fu, Shizhe Zhou, Ligang Liu, and Niloy J Mitra. 2011. Animated construction of line drawings. In Proceedings of the 2011 SIGGRAPH Asia Conference. 1--10.Google ScholarDigital Library
- Yulia Gryaditskaya, Mark Sypesteyn, Jan Willem Hoftijzer, Sylvia C Pont, Frédo Durand, and Adrien Bousseau. 2019. OpenSketch: a richly-annotated dataset of product design sketches. ACM Trans. Graph. 38, 6 (2019), 232--1.Google ScholarDigital Library
- Benoit Guillard, Edoardo Remelli, Pierre Yvernay, and Pascal Fua. 2021. Sketch2Mesh: Reconstructing and Editing 3D Shapes from Sketches. arXiv preprint arXiv:2104.00482 (2021).Google Scholar
- David Ha and Douglas Eck. 2017. A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477 (2017).Google Scholar
- Xiaoguang Han, Chang Gao, and Yizhou Yu. 2017. Deepsketch2face: a deep learning based sketching system for 3d face and caricature modeling. ACM Transactions on graphics (TOG) 36, 4 (2017), 1--12.Google ScholarDigital Library
- Zhe Huang, Hongbo Fu, and Rynson WH Lau. 2014. Data-driven segmentation and labeling of freehand sketches. ACM Transactions on Graphics (TOG) 33, 6 (2014), 1--10.Google ScholarDigital Library
- Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1125--1134.Google ScholarCross Ref
- Tilke Judd, Frédo Durand, and Edward Adelson. 2007. Apparent ridges for line drawing. ACM transactions on graphics (TOG) 26, 3 (2007), 19--es.Google Scholar
- Changjian Li, Hao Pan, Yang Liu, Xin Tong, Alla Sheffer, and Wenping Wang. 2017a. Bendsketch: Modeling freeform surfaces through 2d sketching. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1--14.Google ScholarDigital Library
- Lei Li, Changqing Zou, Youyi Zheng, Qingkun Su, Hongbo Fu, and Chiew-Lan Tai. 2020. Sketch-r2cnn: An rnn-rasterization-cnn architecture for vector sketch recognition. IEEE transactions on visualization and computer graphics (2020).Google Scholar
- Yi Li, Yi-Zhe Song, Timothy M Hospedales, and Shaogang Gong. 2017b. Free-hand sketch synthesis with deformable stroke models. International Journal of Computer Vision 122, 1 (2017), 169--190.Google ScholarCross Ref
- Difan Liu, Mohamed Nabail, Aaron Hertzmann, and Evangelos Kalogerakis. 2020. Neural contours: Learning to draw lines from 3d shapes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 5428--5436.Google ScholarCross Ref
- Yun Liu, Ming-Ming Cheng, Xiaowei Hu, Kai Wang, and Xiang Bai. 2017. Richer convolutional features for edge detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3000--3009.Google ScholarCross Ref
- Zhaoliang Lun, Matheus Gadelha, Evangelos Kalogerakis, Subhransu Maji, and Rui Wang. 2017. 3d shape reconstruction from sketches via multi-view convolutional networks. In 2017 International Conference on 3D Vision (3DV). IEEE, 67--77.Google ScholarCross Ref
- William J Matthews and Amy Adams. 2008. Another reason why adults find it hard to draw accurately. Perception 37, 4 (2008), 628--630.Google ScholarCross Ref
- Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. 2019. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 4460--4470.Google ScholarCross Ref
- Yutaka Ohtake, Alexander Belyaev, and Hans-Peter Seidel. 2004. Ridge-valley lines on meshes via implicit surface fitting. In ACM SIGGRAPH 2004 Papers. 609--612.Google ScholarDigital Library
- Anran Qi, Yulia Gryaditskaya, Jifei Song, Yongxin Yang, Yonggang Qi, Timothy M Hospedales, Tao Xiang, and Yi-Zhe Song. 2021. Toward Fine-Grained Sketch-Based 3D Shape Retrieval. IEEE Transactions on Image Processing 30 (2021), 8595--8606.Google ScholarDigital Library
- Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, and James Hays. 2016. The sketchy database: learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (TOG) 35, 4 (2016), 1--12.Google ScholarDigital Library
- Ryan Schmidt, Azam Khan, Gord Kurtenbach, and Karan Singh. 2009. On expert performance in 3D curve-drawing tasks. In Proceedings of the 6th eurographics symposium on sketch-based interfaces and modeling. 133--140.Google ScholarDigital Library
- Wei Shen, Xinggang Wang, Yan Wang, Xiang Bai, and Zhijiang Zhang. 2015. Deep-contour: A deep convolutional feature learned by positive-sharing loss for contour detection. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3982--3991.Google Scholar
- Wanchao Su, Dong Du, Xin Yang, Shizhe Zhou, and Hongbo Fu. 2018. Interactive sketch-based normal map generation with deep neural networks. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 1 (2018), 1--17.Google ScholarDigital Library
- John Tchalenko. 2009. Segmentation and accuracy in copying and drawing: Experts and beginners. Vision research 49, 8 (2009), 791--800.Google Scholar
- Nanyang Wang, Yinda Zhang, Zhuwen Li, Yanwei Fu, Wei Liu, and Yu-Gang Jiang. 2018. Pixel2mesh: Generating 3d mesh models from single rgb images. In Proceedings of the European conference on computer vision (ECCV). 52--67.Google ScholarDigital Library
- Zeyu Wang, Sherry Qiu, Nicole Feng, Holly Rushmeier, Leonard McMillan, and Julie Dorsey. 2021. Tracing Versus Freehand for Evaluating Computer-Generated Drawings. ACM Trans. Graph. 40, 4 (Aug. 2021), 12 pages. Google ScholarDigital Library
- Saining Xie and Zhuowen Tu. 2015. Holistically-nested edge detection. In Proceedings of the IEEE international conference on computer vision. 1395--1403.Google ScholarDigital Library
- Chuan Yan, David Vanderhaeghe, and Yotam Gingold. 2020. A benchmark for rough sketch cleanup. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--14.Google ScholarDigital Library
- Lumin Yang, Jiajie Zhuang, Hongbo Fu, Xiangzhi Wei, Kun Zhou, and Youyi Zheng. 2021. SketchGNN: Semantic Sketch Segmentation with Graph Neural Networks. ACM Transactions on Graphics (TOG) 40, 3 (2021), 1--13.Google ScholarDigital Library
- Ziv Yaniv, Bradley C Lowekamp, Hans J Johnson, and Richard Beare. 2018. SimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research. Journal of digital imaging 31, 3 (2018), 290--303.Google ScholarCross Ref
- Hui Ye, Kin Chung Kwan, and Hongbo Fu. 2021. 3D curve creation on and around physical objects with mobile AR. IEEE Transactions on Visualization & Computer Graphics 01 (2021), 1--1.Google ScholarDigital Library
- Deng Yu, Lei Li, Youyi Zheng, Manfred Lau, Yi-Zhe Song, Chiew-Lan Tai, and Hongbo Fu. 2020. SketchDesc: Learning local sketch descriptors for multi-view correspondence. IEEE Transactions on Circuits and Systems for Video Technology 31, 5 (2020), 1738--1750.Google ScholarDigital Library
- Qian Yu, Feng Liu, Yi-Zhe Song, Tao Xiang, Timothy M Hospedales, and Chen-Change Loy. 2016. Sketch me that shoe. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 799--807.Google ScholarCross Ref
- Qian Yu, Yongxin Yang, Yi-Zhe Song, Tao Xiang, and Timothy Hospedales. 2015. Sketch-a-net that beats humans. arXiv preprint arXiv:1501.07873 (2015).Google Scholar
- Song-Hai Zhang, Yuan-Chen Guo, and Qing-Wen Gu. 2021. Sketch2Model: View-Aware 3D Modeling from Single Free-Hand Sketches. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 6012--6021.Google ScholarCross Ref
- Yue Zhong, Yulia Gryaditskaya, Honggang Zhang, and Yi-Zhe Song. 2020a. Deep sketch-based modeling: Tips and tricks. In 2020 International Conference on 3D Vision (3DV). IEEE, 543--552.Google ScholarCross Ref
- Yue Zhong, Yonggang Qi, Yulia Gryaditskaya, Honggang Zhang, and Yi-Zhe Song. 2020b. Towards practical sketch-based 3d shape generation: The role of professional sketches. IEEE Transactions on Circuits and Systems for Video Technology 31, 9 (2020), 3518--3528.Google ScholarCross Ref
Index Terms
- DifferSketching: How Differently Do People Sketch 3D Objects?
Recommendations
Tracing versus freehand for evaluating computer-generated drawings
Non-photorealistic rendering (NPR) and image processing algorithms are widely assumed as a proxy for drawing. However, this assumption is not well assessed due to the difficulty in collecting and registering freehand drawings. Alternatively, tracings ...
LoFi Sketch: A Large Scale Dataset of Smartphone Low Fidelity Sketches
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing SystemsRecent research on automating the transformation of low fidelity (LoFi) sketches to code using Deep Neural Networks require a large-scale dataset for generalizable results. This paper introduces the LoFi sketch dataset, an open-access dataset of hand-...
Stroke-based semantic segmentation for scene-level free-hand sketches
AbstractSketching is a simple and efficient way for humans to express their perceptions of the world. Sketch semantic segmentation plays a key role in sketch understanding and is widely used in sketch recognition, sketch-based image retrieval, or editing. ...
Comments