A study on the automatic generation of banner layouts
Graphical abstract
Introduction
In modern life, banners appear in various medias such as advertisements, websites, and mobile phones since they play a key role in visual communication of various messages. Creating an original design is more artistic and some design principles should be complied to achieve pleasing visual appearance. Designing a beautiful and good quality banner is a difficult and time-consuming task, to refine it, since such tasks require professional skills to convey information clearly while also satisfying aesthetic goals. Moreover, many banners are created by non-experts, which consumes a lot of time and the quality is unsatisfying. As the need of creating new banners keeps growing rapidly, automatic tools for creating, adapting, and improving banners design could not only release the effort of designers, but also encourage new ideas and educate novice users.
A banner is often composed of an underlying theme, a background image, the brand logo in focus and affiliated text phrases [1]. These elements are usually taken from a library, placed according to a design layout and applied few image transformations to create a banner (Fig. 1). For designers, a layout is a blueprint gathering design elements to guide them to create design works. It determines the spatial arrangement of various design elements like text, logo, and others. It can be used to describe a group of banners, and to generate new banners with the same blueprint. Following design principles, it is composed of a set of rectangles, which determines the position and size of a design element. A high-quality layout can generate aesthetic pleasing banners and thus enhance information presentation, guide reader attention [2]. In recent years, design layout generation has received a growing interest in computer graphic community. Prior works focusing on layout generation are involved with design style, aesthetic measures, and perception [3,4].
Banner design is a complex problem involving layout, colors, fonts, design intent, and requirements gathering. In this work, however, we specifically focus on the layout generation. The layout generation is defined as specifying the locations and sizes of all elements in a banner. In this work, we assume that a set of texts and graphic elements are provided as inputs along with associated meta-data and their corresponding roles. The goal is to generate a visually satisfying banner by arranging these elements properly according to design principles. Although banner design involves a set of choices including colors, fonts and lines breaks, we leave selecting these variables for future work.
We present an automatic tool capable of creating and improving design layouts for design elements of arbitrary sizes. Due to the complexity of banner design, our approach has two stages. Given a set of design elements and their corresponding roles, we first initialize design layouts according to elements’ sizes. After that, we use energy terms to evaluate the quality of initial layouts and improve those with low quality. Since we do not consider elements’ style, color etc. in a design process, our quality evaluation metrics is different from those described in [6], which takes the darkness, brightness, HSV distribution of design elements into account. Finally, we introduce two methods for layout improvement. One is based on designers’ experience of adjusting bad designs. The other one is based on reinforcement learning, where agent learns the policy of layout optimization through interacting with environment. We believe this tool enables to assist designers to create new ideas and process large amount of designs in their daily work.
Our key technical contributions include:
- •
We propose a traditional method that is used by designers to adjust design layouts according to design principles. Actions for adjusting positions of design elements are transformed into algorithms separately.
- •
A deep reinforcement learning based method with two different reward functions for layout generation is introduced. Under our evaluation metrics, experiments are conducted to test the performance of different methods.
We test the proposed methods on real banner designs, producing competitive results with those created by professional designers. In the rest of this paper, we will first investigate prior work in Section 2 and clarify layout generation and evaluation in Section 3, then describe how to optimize design layouts in Section 4. Finally, we show an experimental study of proposed methods in Section 5 and conclude this work in Section 6.
Section snippets
Related works
In this section, approaches of automatic layout generation are discussed. They are classified into learning-based methods, constraint-based methods, and probabilistic graphical model-based methods.
A state-of-the-art learning approaches have been proposed to generate multi-size banners automatically by utilizing a triangle interpolation with learned style parameters and a multi-size style transfer technique [7]. LayoutGAN is used to synthesize layout by modeling geometric relations of different
Layout generation
This section describes our approach of generating layouts for a given set of design elements. The main idea is to initialize the position of bounding boxes of design elements of a layout and then optimize the layout with our approaches.
Layout optimization
In this section, we propose two methods to optimize the initial layouts based on evaluation metrics described above.
The first one is based on designer's experience of moving elements where algorithms are proposed for moving actions. The other one is based on reinforcement learning where DDPG [16] with different reward functions is introduced.
Training
We train the model on a dataset with 1800 layouts. Positions of bounding boxes within a layout are initialized randomly. The implementation details are as following. We simulate 500000 episodes. Each episode is limited to 500ms, which means there are at most 200 steps in a single episode. Initial state for each episode is sampled randomly in the state space. The learning rate is set to 0.00025 and the discounted factor is set to 0.99. The batch size is set to 128. The capacity of the transition
Conclusion
In this paper, we propose methods to automatically generate layouts of design banners. With the purpose of generating a design layout satisfying aesthetic measures, a layout is initialized by randomly positioning design elements, and then reorganize them with proposed methods. Proposed methods are tested and compared on a given banner dataset. Results of experiments show that the deep reinforcement learning based method outperforms the one of traditional method with respect to aesthetic
Author statement
Hao Hu: Conceptualization, Methodology, Software, Writing- Original draft preparation
Chao Zhang: Data curation, Validation, Visualization, Investigation.
Yanxue Liang: Writing- Reviewing and Editing, Supervision.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Hao Hu received the PhD degree from Arts et Metiers ParisTech, France, in 2018. He is a senior researcher in Westlake University. His research interests include computer vision, perception, computer aided design and manufacturing.
References (24)
- et al.
Over-constraints detection and resolution in geometric equation systems
Comput-Aided Des
(2017) - Vempati, Sreekanth, and Korah T. Malayil. “Enabling hyper-personalization: Automated ad creative generation and ranking...
- et al.
Look over here: Attention-directing composition of manga elements
ACM Trans Graph
(July 2014) - et al.
Probabilistic document model for automated document composition
- et al.
Automatic generation of visual-textual presentation layout
ACM Trans Multimedia Comput, Commun Appl (TOMM)
(2016) Learning Design: Aesthetic Models for Color, Layout, and Typography
(2015)- et al.
Monet: A system for reliving your memories by theme-based photo storytelling
IEEE Trans Multimedia
(2016) - et al.
Layout style modeling for automating banner design
- J. Li, J. Yang, A. Hertzmann, J. Zhang, and T. Xu, “Layoutgan: Generating graphic layouts with wireframe...
- et al.
Facility layout design using virtual multi-agent system
J Intell Manuf
(2009)
A q-learning system for container marshalling with group-based learning model at container yard terminals
Learning layouts for single-page graphic designs
IEEE Trans Vis Comput Graph
Cited by (0)
Hao Hu received the PhD degree from Arts et Metiers ParisTech, France, in 2018. He is a senior researcher in Westlake University. His research interests include computer vision, perception, computer aided design and manufacturing.
Chao Zhang received his master's degree from Chang'An University in 2019. He is a computer vision engineer in Intelligent Industry Research Institute of Westlake University. His research interests include Deep Learning and Reinforcement Learning applications.
Yanxue Liang Received his master's degree from Shanghai JiaoTong University in 2003 and PhD degree from Tokyo Institute of Technology in 2008. He is now a professor in Westlake University. His current research interests include perception of robotics, CAD, and CAM.
This paper is for special section VSI-hci. Reviews processed and recommended for publication by Guest Editor Dr. Shengzong Zhou.
This paper was recommended for publication by Associate Editor Dr. M. Malek.
- 1
Equal contribution