Sukanya Bhattacharjee
ABSTRACT
Sketching is one of the most natural ways for representing any object pictorially. With the advent of Virtual Reality (VR) and Augmented Reality (AR) technologies, 3D sketching has become more accessible. But it is challenging to convert these sketches to 3D models in a manner consistent with the intent of the artist. Learning based methods provide an effective alternate paradigm for solving this classical problem of sketch based modelling in interactive platforms. Surface patches, inferred from 3D sketch strokes, can be treated as a primitive which are assembled to create complex object models. We present our proposed framework for this problem and discuss our solution to one of its core challenges. Our deep neural network based method allows users to create surfaces from a stream of sparse 3D sketch strokes. We also show integration of our method into an existing Blender based 3D content creation pipeline. This serves as a basis to solve a more complex problem of creating complete 3D object models from sparse 3D sketch strokes.
Supplemental Material
Available for Download
- Alexis Andre and Suguru Saito. 2011. Single-view sketch based modeling. In Proceedings of the Eighth Eurographics Symposium on Sketch-Based Interfaces and Modeling. ACM, 133–140.Google Scholar
Digital Library
- Seok-Hyung Bae, Ravin Balakrishnan, and Karan Singh. 2008. ILoveSketch: as-natural-as-possible sketching system for creating 3d curve models. In Proceedings of the 21st annual ACM symposium on User interface software and technology. ACM, 151–160.Google ScholarDigital Library
- Jonathan T Barron. 2019. A general and adaptive robust loss function. In Proc. of CVPR. 4331–4339.Google ScholarCross Ref
- Harry G Barrow, Jay M Tenenbaum, Robert C Bolles, and Helen C Wolf. 1977. Parametric correspondence and chamfer matching: Two new techniques for image matching. Technical Report. SRI International Menlo Park CA AI Center.Google Scholar
- Sukanya Bhattacharjee and Parag Chaudhuri. 2020. A Survey on Sketch Based Content Creation: from the Desktop to Virtual and Augmented Reality. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 757–780.Google Scholar
- Sukanya Bhattacharjee and Parag Chaudhuri. 2022. Deep Interactive Surface Creation from 3D Sketch Strokes. In Proceedings of IJCAI, Lud De Raedt (Ed.). International Joint Conferences on Artificial Intelligence Organization, 4908–4914. AI and Arts Track.Google ScholarCross Ref
- Onur Rauf Bingol and Adarsh Krishnamurthy. 2019. NURBS-Python: An open-source object-oriented NURBS modeling framework in Python. SoftwareX 9 (2019), 85–94.Google ScholarCross Ref
- Blender. 2017. Blender 2.79b. https://www.blender.org/ Last accessed on 15-10-2019.Google Scholar
- Chris De Paoli and Karan Singh. 2015. SecondSkin: sketch-based construction of layered 3D models. ACM Transactions on Graphics (TOG) 34, 4 (2015), 126.Google ScholarDigital Library
- Michael F Deering. 1995. HoloSketch: a virtual reality sketching/animation tool. ACM Transactions on Computer-Human Interaction (TOCHI) 2, 3 (1995), 220–238.Google ScholarDigital Library
- Even Entem, Loïc Barthe, Marie-Paule Cani, Frederic Cordier, and Michiel Van De Panne. 2015. Modeling 3D animals from a side-view sketch. Computers & Graphics 46 (2015), 221–230.Google ScholarDigital Library
- Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. 2019. Abc: A big cad model dataset for geometric deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9601–9611.Google ScholarCross Ref
- Zhihao Liu, Fanxing Zhang, and Zhanglin Cheng. 2021. BuildingSketch: Freehand Mid-Air Sketching for Building Modeling. In 2021 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 329–338.Google Scholar
- Zhongjin Luo, Jie Zhou, Heming Zhu, Dong Du, Xiaoguang Han, and Hongbo Fu. 2021. Simpmodeling: Sketching implicit field to guide mesh modeling for 3d animalmorphic head design. In The 34th Annual ACM Symposium on User Interface Software and Technology. 854–863.Google ScholarDigital Library
- Andrew Nealen, Takeo Igarashi, Olga Sorkine, and Marc Alexa. 2007. FiberMesh: designing freeform surfaces with 3D curves. In ACM SIGGRAPH 2007 papers. ACM, 41–50.Google ScholarDigital Library
- QI Quan and LI Qingde. [n. d.]. Thin Implicit Utah Teapot: Design for Additive Manufacturing. representations 1 ([n. d.]), 2.Google Scholar
- Enrique Rosales, Jafet Rodriguez, and Alla Sheffer. 2019. SurfaceBrush: from virtual reality drawings to manifold surfaces. ACM Trans. Graph. 38, 4 (2019), 96:1–96:15.Google ScholarDigital Library
- Gopal Sharma, Difan Liu, Subhransu Maji, Evangelos Kalogerakis, Siddhartha Chaudhuri, and Radomír Měch. 2020. Parsenet: A parametric surface fitting network for 3d point clouds. In European Conference on Computer Vision. Springer, 261–276.Google ScholarDigital Library
- Daniel Sỳkora, Ladislav Kavan, Martin Čadík, Ondřej Jamriška, Alec Jacobson, Brian Whited, Maryann Simmons, and Olga Sorkine-Hornung. 2014. Ink-and-ray: Bas-relief meshes for adding global illumination effects to hand-drawn characters. ACM Transactions on Graphics (TOG) 33, 2 (2014), 1–15.Google ScholarDigital Library
- Min Xin, Ehud Sharlin, and Mario Costa Sousa. 2008. Napkin sketch: handheld mixed reality 3D sketching. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology. ACM, 223–226.Google ScholarDigital Library
- Baoxuan Xu, William Chang, Alla Sheffer, Adrien Bousseau, James McCrae, and Karan Singh. 2014. True2Form: 3D curve networks from 2D sketches via selective regularization. ACM Transactions on Graphics 33, 4 (2014), 131:1–131:13.Google ScholarDigital Library
- Pengfei Xu, Hongbo Fu, Youyi Zheng, Karan Singh, Hui Huang, and Chiew-Lan Tai. 2018. Model-Guided 3D Sketching. IEEE Transactions on Visualization and Computer Graphics (2018), 2927–2939.Google Scholar
- Yaoqing Yang, Chen Feng, Yiru Shen, and Dong Tian. 2018. Foldingnet: Point cloud auto-encoder via deep grid deformation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 206–215.Google ScholarCross Ref
- Emilie Yu, Rahul Arora, J Andreas Baerentzen, Karan Singh, and Adrien Bousseau. 2022. Piecewise-smooth surface fitting onto unstructured 3D sketches. ACM Transactions on Graphics (TOG) 41, 4 (2022), 1–16.Google ScholarDigital Library
- Emilie Yu, Rahul Arora, Tibor Stanko, J Andreas Bærentzen, Karan Singh, and Adrien Bousseau. 2021. Cassie: Curve and surface sketching in immersive environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14.Google ScholarDigital Library
- Fanxing Zhang, Zhihao Liu, Zhanglin Cheng, Oliver Deussen, Baoquan Chen, and Yunhai Wang. 2021. Mid-air finger sketching for tree modeling. In 2021 IEEE Virtual Reality and 3D User Interfaces (VR). IEEE, 826–834.Google Scholar
Index Terms
- Learning Assisted Interactive 3D modelling from 3D sketches
Comments