Guolong Wang
ABSTRACT
Selecting elements for graphic design is essential for ensuring a correct understanding of clients' requirements as well as improving the efficiency of designers before a fine-designed process. Some semi-automatic design tools proposed layout templates where designers always select elements according to the rectangular boxes that specify how elements are placed. In practice, layout and element selection are complementary. Compared to the layout which can be readily obtained from pre-designed templates, it is generally time-consuming to mindfully pick out suitable elements, which calls for an automation of elements selection. To address this, we formulate element selection as a sequential decision-making process and develop a deep element selection network (DESN). Given a layout file with annotated elements, new graphical elements are selected to form graphic designs based on aesthetics and consistency criteria. To train our DESN, we propose an end-to-end, reinforcement learning based framework, where we design a novel reward function that jointly accounts for visual aesthetics and consistency. Based on this, visually readable and aesthetic drafts can be efficiently generated. We further contribute a layout-poster dataset with exhaustively labeled attributes of poster key elements. Qualitative and quantitative results indicate the efficacy of our approach.
- Jaime Arguello and Bogeum Choi. 2019. The Effects of Working Memory, Perceptual Speed, and Inhibition in Aggregated Search. TOIS, Vol. 37, 3 (2019), 1--34.Google Scholar
Digital Library
- Rudolf Arnheim. 1956. Art and visual perception: a psychology of the creative eye. Philosophy & Phenomenological Research, Vol. 16, 3 (1956), 425.Google ScholarCross Ref
- Tao Chen, Ming-Ming Cheng, Ping Tan, Ariel Shamir, and Shi-Min Hu. 2009. Sketch2photo: Internet image montage. TOG, Vol. 28, 5 (2009), 124.Google ScholarDigital Library
- Wengling Chen and James Hays. 2018. Sketchygan: Towards diverse and realistic sketch to image synthesis. In CVPR. 9416--9425.Google Scholar
- Niranjan Dameravenkata, Josep Bento, and Eamonn Obrienstrain. 2011. Probabilistic document model for automated document composition. (2011), 3--12.Google Scholar
- Jarrod Gingras. [n.d.]. https://www.realstorygroup.com/Blog/3220-Do-You-Need-Digital-Asset-Management? https://www.realstorygroup.com/Blog/3220-Do-You-Need-Digital-Asset-Management?.Google Scholar
- Shixiang Gu, Timothy P Lillicrap, Ilya Sutskever, and Sergey Levine. 2016. Continuous deep Q-learning with model-based acceleration. In ICML. 2829--2838.Google Scholar
- Robert M Haralick. 2005. Statistical and structural approaches to texture. Proc. IEEE, Vol. 67, 5 (2005), 786--804.Google ScholarCross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.Google Scholar
- Elad Hoffer, Itay Hubara, and Daniel Soudry. 2017. Train longer, generalize better: closing the generalization gap in large batch training of neural networks. NeurIPS, 1731--1741.Google Scholar
- Xiansheng Hua. 2018. Challenges and Practices of Large Scale Visual Intelligence in the Real-World. In ACM MM. 364--364.Google Scholar
- Nathan Hurst, Wilmot Li, and Kim Marriott. 2009. Review of automatic document formatting. In Proceedings of the 9th ACM symposium on Document engineering. 99--108.Google ScholarDigital Library
- Phillip Isola, Jun Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In CVPR. 5967--5976.Google Scholar
- Yu Liu, Tao Mei, and Chang Wen Chen. 2016. Automatic suggestion of presentation image for storytelling. In ICME. 1--6.Google Scholar
- Xin Lu, Zhe Lin, Hailin Jin, Jianchao Yang, and James Z Wang. 2014. Rapid: Rating pictorial aesthetics using deep learning. In ACMMM. 457--466.Google ScholarDigital Library
- Yiwen Luo and Xiaoou Tang. 2008. Photo and Video Quality Evaluation: Focusing on the Subject. In ECCV.Google Scholar
- Jana Machajdik and Allan Hanbury. 2010. Affective image classification using features inspired by psychology and art theory. In ACM MM. 83--92.Google Scholar
- Luana Micallef, Gregorio Palmas, Antti Oulasvirta, and Tino Weinkauf. 2017. Towards Perceptual Optimization of the Visual Design of Scatterplots. TVCG, Vol. 23, 6 (2017), 1588--1599.Google ScholarDigital Library
- Naila Murray, Luca Marchesotti, and Florent Perronnin. 2012. AVA: A large-scale database for aesthetic visual analysis. In CVPR. 2408--2415.Google Scholar
- Alejandro Newell, Kaiyu Yang, and Jia Deng. 2016. Stacked Hourglass Networks for Human Pose Estimation. In ECCV. 483--499.Google Scholar
- Peter Odonovan, Aseem Agarwala, and Aaron Hertzmann. 2014. Learning Layouts for Single-PageGraphic Designs. TVCG, Vol. 20, 8 (2014), 1200--1213.Google ScholarDigital Library
- Peter Odonovan, Aseem Agarwala, and Aaron Hertzmann. 2015. DesignScape: Design with Interactive Layout Suggestions. In ACM CHI. 1221--1224.Google Scholar
- Xufang Pang, Ying Cao, Rynson W H Lau, and Antoni B Chan. 2016. Directing user attention via visual flow on web designs. ACM Transactions on Graphics (TOG), Vol. 35, 6 (2016), 240.Google ScholarDigital Library
- Yuting Qiang, Yanwei Fu, Yanwen Guo, Zhihua Zhou, and Leonid Sigal. 2016. Learning to generate posters of scientific papers. In AAAI. 51--57.Google Scholar
- Filip Radenović, Giorgos Tolias, and Ondvr ej Chum. 2016. CNN image retrieval learns from BoW: Unsupervised fine-tuning with hard examples. In European conference on computer vision. Springer, 3--20.Google ScholarCross Ref
- Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S Bernstein, et al. 2015. ImageNet Large Scale Visual Recognition Challenge. IJCV, Vol. 115, 3 (2015), 211--252.Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR.Google Scholar
- Hsiao-Hang Su, Tse-Wei Chen, Chieh-Chi Kao, Winston H Hsu, and Shao-Yi Chien. 2011. Scenic photo quality assessment with bag of aesthetics-preserving features. In ACM MM. 1213--1216.Google Scholar
- Hossein Talebi and Peyman Milanfar. 2017. NIMA: Neural Image Assessment. TIP, Vol. PP, 99 (2017), 1--1.Google Scholar
- Xin-Yi Tong, Gui-Song Xia, Fan Hu, Yanfei Zhong, Mihai Datcu, and Liangpei Zhang. 2019. Exploiting deep features for remote sensing image retrieval: A systematic investigation. IEEE Transactions on Big Data (2019).Google Scholar
- George Tucker, Surya Bhupatiraju, Shixiang Gu, Richard E Turner, Zoubin Ghahramani, and Sergey Levine. 2018. The Mirage of Action-Dependent Baselines in Reinforcement Learning. In ICML. 5015--5024.Google Scholar
- Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. 2019. Composing text and image for image retrieval-an empirical odyssey. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6439--6448.Google ScholarCross Ref
- Liwei Wang, Yin Li, and Svetlana Lazebnik. 2016. Learning Deep Structure-Preserving Image-Text Embeddings. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5005--5013.Google ScholarCross Ref
- Yan Wang, Tao Mei, and Xian Sheng Hua. 2011. Community Discovery from Movie and Its Application to Poster Generation. Springer Berlin Heidelberg. 107--117 pages.Google Scholar
- wikihow. [n.d.]. Design-Posters. https://www.wikihow.com/Design-Posters.Google Scholar
- Shuai Yang, Jiaying Liu, Wenhan Yang, and Zongming Guo. 2018. Context-Aware Text-Based Binary Image Stylization and Synthesis. TIP, Vol. PP, 99 (2018), 1--1.Google Scholar
- Sasi Kiran Yelamarthi, M Shiva Krishna Reddy, Ashish Kumar Mishra, and Anurag Mittal. 2018. A Zero-Shot Framework for Sketch Based Image Retrieval. In ECCV. 316--333.Google Scholar
- Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2016. Building a large scale dataset for image emotion recognition: the fine print and the benchmark. In AAAI. 308--314.Google Scholar
- Nanxuan Zhao, Ying Cao, and Rynson WH Lau. 2018. What characterizes personalities of graphic designs? TOG, Vol. 37, 4 (2018), 116.Google ScholarDigital Library
- Xinru Zheng, Xiaotian Qiao, Ying Cao, and Rynson WH Lau. 2019. Content-aware generative modeling of graphic design layouts. ACM Transactions on Graphics (TOG), Vol. 38, 4 (2019), 1--15.Google ScholarDigital Library
Index Terms
- Learning to Select Elements for Graphic Design
Recommendations
The Enriched Crouzeix---Raviart Elements are Equivalent to the Raviart---Thomas Elements
For both the Poisson model problem and the Stokes problem in any dimension, this paper proves that the enriched Crouzeix---Raviart elements are actually identical to the first order Raviart---Thomas elements in the sense that they produce the same ...
Mortar Boundary Elements
We establish a mortar boundary element scheme for hypersingular boundary integral equations representing elliptic boundary value problems in three dimensions. We prove almost quasi-optimal convergence of the scheme in broken Sobolev norms of order $1/2$...
Optimal superconvergence analysis for the Crouzeix-Raviart and the Morley elements
AbstractIn this paper, an improved superconvergence analysis is presented for both the Crouzeix-Raviart element and the Morley element. The main idea of the analysis is to employ a discrete Helmholtz decomposition of the difference between the canonical ...
Comments