Haocheng Ren
Skip Abstract Section
Skip Supplemental Material SectionAbstract
Conventionally, interior lighting design is technically complex yet challenging and requires professional knowledge and aesthetic disciplines of designers. This article presents a new digital lighting design framework for virtual interior scenes, which allows novice users to automatically obtain lighting layouts and interior rendering images with visually pleasing lighting effects. The proposed framework utilizes neural networks to retrieve and learn underlying design guidelines and the principles beneath the existing lighting designs, e.g., a newly constructed dataset of 6k 3D interior scenes from professional designers with dense annotations of lights. With a 3D furniture-populated indoor scene as the input, the framework takes two stages to perform lighting design: (1) lights are iteratively placed in the room; (2) the colors and intensities of the lights are optimized by an adversarial scheme, resulting in lighting designs with aesthetic lighting effects. Quantitative and qualitative experiments show that the proposed framework effectively learns the guidelines and principles and generates lighting designs that are preferred over the rule-based baseline and comparable to those of professional human designers.
Supplemental Material
Available for Download
pdf
3582001-supp.pdf (12.1 MB)
Supplementary material
- . 2021. 3ds Max. Retrieved from https://www.autodesk.com/products/3ds-max/overview.Google Scholar
- . 2019. Scan2CAD: Learning CAD model alignment in RGB-D scans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2614–2623.Google ScholarCross Ref
- . 2019. Deep CG2Real: Synthetic-to-real translation via image disentanglement. In Proceedings of the IEEE International Conference on Computer Vision. 2730–2739.Google ScholarCross Ref
- . 2014. Digital Lighting & Rendering. Pearson Education.Google Scholar
- . 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 97–104.Google Scholar
- . 2019. AI + Architecture: Towards a New Approach. Master’s thesis. Harvard School of Design.Google Scholar
- . 2015. Magic decorator: Automatic material suggestion for indoor digital scenes. ACM Trans. Graph. 34, 6 (2015), 1–11.Google ScholarDigital Library
- . 2018. Deep photo enhancer: Unpaired learning for image enhancement from photographs with GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6306–6314.Google ScholarCross Ref
- . 2022. Coohom. Retrieved from https://www.coohom.com/.Google Scholar
- . 1986. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statist. Sci. Feb. (1986), 54–75.Google Scholar
- . 2021. 3D-FRONT: 3D furnished rooms with layOuts and semaNTics. In Proceedings of the IEEE International Conference on Computer Vision. 10933–10942.Google ScholarCross Ref
- . 2014. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014), 2672–2680.Google ScholarDigital Library
- . 2015. Interior Lighting for Designers. John Wiley & Sons.Google Scholar
- . 1983. The decomposition of polygons into convex parts. Computat. Geom. 1 (1983), 235–259.Google Scholar
- . 2016a. Understanding real world indoor scenes with synthetic data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4077–4085.Google Scholar
- . 2016b. SceneNet: An annotated model generator for indoor scene understanding. In Proceedings of the IEEE International Conference on Robotics and Automation. IEEE, 5737–5743.Google ScholarDigital Library
- . 2020. Graph2Plan: Learning floorplan generation from layout graphs. ACM Trans. Graph. 39, 4 (2020), 118–1.Google ScholarDigital Library
- . 2018. Exposure: A white-box photo post-processing framework. ACM Trans. Graph. 37, 2 (2018), 1–17.Google ScholarDigital Library
- . 2017. DSLR-quality photos on mobile devices with deep convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision. 3277–3285.Google ScholarCross Ref
- . 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1125–1134.Google ScholarCross Ref
- . 2022. DR. JIT: A just-in-time compiler for differentiable rendering. ACM Trans. Graph. 41, 4 (2022), 1–19.Google ScholarDigital Library
- . 2019. Lighting layout optimization for 3D indoor scenes. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 733–743.Google Scholar
- . 2016. Perceptual losses for real-time style transfer and super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 694–711.Google ScholarCross Ref
- . 2020. Analyzing and improving the image quality of StyleGAN. In Proceedings of the IEEE International Conference on Computer Vision.Google ScholarCross Ref
- . 1993. Radioptimization: Goal based rendering. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques. 147–154.Google ScholarDigital Library
- . 2009. Toward evaluating lighting design interface paradigms for novice users. ACM Trans. Graph. 28, 3 (2009), 1–9.Google ScholarDigital Library
- . 2017. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning. 1857–1865.Google ScholarDigital Library
- . 2019. GRAINS: Generative Recursive Autoencoders for Indoor Scenes. ACM Trans. Graph. 38, 2 (2019), 1–16.Google ScholarDigital Library
- . 2018. Differentiable Monte Carlo ray tracing through edge sampling. ACM Trans. Graph. 37, 6 (2018), 1–11.Google ScholarDigital Library
- . 2021. OpenRooms: An open framework for photorealistic indoor scene datasets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7190–7199.Google ScholarCross Ref
- . 2013. Interactive lighting design with hierarchical light representation. In Computer Graphics Forum, Vol. 32. Wiley Online Library, 133–142.Google Scholar
- . 2015. Style compatibility for 3D furniture models. ACM Trans. Graph. 34, 4 (2015), 1–9.Google ScholarDigital Library
- . 1997. Design galleries: A general approach to setting parameters for computer graphics and animation. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. 389–400.Google ScholarDigital Library
- . 2021. Maya. Retrieved from https://www.autodesk.com/products/maya/overview.Google Scholar
- . 2011. Interactive furniture layout using interior design guidelines. ACM Trans. Graph. 30, 4 (2011), 1–10.Google ScholarDigital Library
- . 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).Google Scholar
- . 2012. AVA: A large-scale database for aesthetic visual analysis. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2408–2415.Google ScholarCross Ref
- . 1965. A simplex method for function minimization. Comput. J. 7, 4 (1965), 308–313.Google ScholarCross Ref
- . 2019. Semantic image synthesis with spatially-adaptive normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2337–2346.Google ScholarCross Ref
- . 2021. ATISS: Autoregressive Transformers for Indoor Scene Synthesis. Adv. Neural Inf. Process. Syst. 34 (2021).Google Scholar
- . 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems. 32, (2019).Google Scholar
- . 2007. Lighting with paint. ACM Trans. Graph. 26, 2 (2007), 9–es.Google ScholarDigital Library
- . 2002. A user interface for interactive cinematic shadow design. ACM Trans. Graph. 21, 3 (2002), 563–566.Google ScholarDigital Library
- . 2022. Planner5D. Retrieved from https://www.planner5d.com/.Google Scholar
- . 2022. MINERVAS: Massive interior environments virtual synthesis. In Computer Graphics Forum, Vol. 41. Wiley Online Library, 63–74.Google Scholar
- . 2019. Fast and flexible indoor scene synthesis via deep convolutional generative models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6182–6190.Google ScholarCross Ref
- . 2021. Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding. In Proceedings of the IEEE International Conference on Computer Vision. 10912–10922.Google ScholarCross Ref
- . 1993. Painting with light. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques. 143–146.Google ScholarDigital Library
- . 2014. Procedural design of exterior lighting for buildings with complex constraints. ACM Trans. Graph. 33, 5 (2014), 1–16.Google ScholarDigital Library
- . 2001. Automatic lighting design using a perceptual quality metric. In Computer Graphics Forum, Vol. 20. Wiley Online Library, 215–227.Google Scholar
- . 2009. Image appearance exploration by model-based navigation. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 629–638.Google Scholar
- . 2019. Exploratory stage lighting design using visual objectives. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 417–429.Google Scholar
- . 2012. Indoor segmentation and support inference from RGBD images. In Proceedings of the European Conference on Computer Vision (ECCV). Springer, 746–760.Google ScholarDigital Library
- . 2017. Semantic scene completion from a single depth image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1746–1754.Google ScholarCross Ref
- . 2021. Learning view selection for 3D scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 14464–14473.Google ScholarCross Ref
- . 2018. NIMA: Neural image assessment. IEEE Trans. Image Process. 27, 8 (2018), 3998–4011.Google ScholarCross Ref
- . 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 11 (2008).Google Scholar
- . 2021. VRay.Retrieved from https://www.chaosgroup.com/.Google Scholar
- . 2019. LightGuider: Guiding interactive lighting design using suggestions, provenance, and quality visualization. IEEE Trans. Visualiz. Comput. Graph. 26, 1 (2019), 569–578.Google Scholar
- . 2019. PlanIT: Planning and instantiating indoor scenes with relation graph and spatial prior networks. ACM Trans. Graph. 38, 4 (2019), 1–15.Google ScholarDigital Library
- . 2018b. Deep convolutional priors for indoor scene synthesis. ACM Trans. Graph. 37, 4 (2018), 1–14.Google ScholarDigital Library
- . 2018a. High-resolution image synthesis and semantic manipulation with conditional GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- . 2021. SceneFormer: Indoor scene generation with transformers. In Proceedings of the International Conference on 3D Vision (3DV). IEEE, 106–115.Google ScholarCross Ref
- . 2018. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 606–615.Google ScholarCross Ref
- . 2019. Data-driven interior plan generation for residential buildings. ACM Trans. Graph. 38, 6 (2019), 1–12.Google ScholarDigital Library
- . 2019. Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation. ACM Trans. Graph. 38, 6 (2019), 224–1.Google ScholarDigital Library
- . 2014. Organizing heterogeneous scene collections through contextual focal points. ACM Trans. Graph. 33, 4 (2014), 1–12.Google ScholarDigital Library
- . 2016. Automatic photo adjustment using deep neural networks. ACM Trans. Graph. 35, 2 (2016), 1–15.Google ScholarDigital Library
- . 2017. DualGAN: Unsupervised dual learning for image-to-image translation. In Proceedings of the IEEE International Conference on Computer Vision. 2849–2857.Google ScholarCross Ref
- . 2011. Make it home: Automatic optimization of furniture arrangement. ACM Trans. Graph. 30, 4 (2011).Google ScholarDigital Library
- . 2020a. Path-space differentiable rendering. ACM Trans. Graph. 39, 4 (2020).Google ScholarDigital Library
- . 2020b. Deep generative modeling for scene synthesis via hybrid representations. ACM Trans. Graph. 39, 2 (2020), 1–21.Google ScholarDigital Library
- . 2017a. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 2223–2232.Google ScholarCross Ref
- . 2017b. Toward multimodal image-to-image translation. In Proceedings of the International Conference on Advances in Neural Information Processing Systems.Google Scholar
Recommendations
Precomputed local radiance transfer for real-time lighting design
This paper introduces a new method for real-time relighting of scenes illuminated by local light sources. We extend previous work on precomputed radiance transfer for distant lighting to local lighting by introducing the concept of unstructured light ...
Precomputed local radiance transfer for real-time lighting design
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersThis paper introduces a new method for real-time relighting of scenes illuminated by local light sources. We extend previous work on precomputed radiance transfer for distant lighting to local lighting by introducing the concept of unstructured light ...
A programmable system for artistic volumetric lighting
We present a method for generating art-directable volumetric effects, ranging from physically-accurate to non-physical results. Our system mimics the way experienced artists think about volumetric effects by using an intuitive lighting primitive, and ...
Comments