Quantitative analysis of human facial beauty using geometric features

Abstract

Perception of human facial beauty is an important aspect of human intelligence and has attracted interests of researchers from diverse fields such as psychology and computer science. Previous studies, however, have the following limitations. First, they did not well quantify the facial feature space. Second, they seldom consider the transformation occurring to faces or the physical sizes of faces. Third, most of them require intensive manual work, e.g. marking landmarks. To overcome these limitations, this paper maps faces onto a human face shape space, and then quantitatively analyses the effect of facial geometric features on human facial beauty by using a similarity transformation invariant shape distance measurement and advanced automatic image processing techniques. With the proposed methodology, we experiment on tens of thousands of female and male faces, revealing that human face shapes lie in a very compact region of the geometric feature space and that female and male average face shapes are very similar. Further, we demonstrate that a face can become more beautiful by making its geometric feature getting obviously closer to the average face shape, but if its distance to the average face shape is already relatively small, deforming it further toward the average face shape cannot effectively improve its attractiveness.

Introduction

Human facial beauty or attractiveness has stimulated the concerted efforts of researchers from diverse fields such as perception, psychology, biology, artificial intelligence, and so on. People became aware of facial beauty even in ancient times, and throughout the long history of human beings, many rules of thumb have been summarized as to what kind of faces looks more attractive [1], [2], [3], [4], [5], [6]. For example, faces with the following features are thought to be more attractive: neonate features such as a small nose and high forehead, or mature features such as prominent cheekbones, or expressive features such as arched eyebrows. Recent psychological studies have found that there is a considerable amount of agreement across age, sex, ethnicity, and social class as to what is a beautiful face [7], [8], [9], and it has been demonstrated that a number of factors underlie the perception of facial attractiveness, including facial skin texture, hair color, symmetry, and the size, shape, and location of organs on the face, and so on [1], [2], [3], [4], [5], [6], [7], [8], [9]. In the past several decades, the advent of computers and the rapid development of computer image and graphics technology as well as computational intelligence technology greatly facilitate and advance the study on facial beauty [10], [11], [12], [13], [14], [15], [16], [17], [18]. For example, genetic algorithm has been adopted to generate prototypes of beautiful faces [10]; image morphing was applied to calculating the average face from a set of faces and to changing the appearance of face [11], [12]; and advanced machine learning techniques such as support vector regression (SVR) have been used to construct a machine predictor of facial beauty and to automatically do beautification for faces [13], [14], [15].

Geometric feature based facial beauty analysis attracted the attention of researchers from psychology and computer science. Landmarks are the most frequently used features to represent the geometric structure of a face [8], [13], [14], [15], [17], [24]. However, different works used different definitions of landmarks (e.g. 224 landmarks in [8], 37 landmarks in [13], 84 landmarks in [15], 29 landmarks in [17], 173 landmarks in [24]), and most of them manually marked the landmarks. The datasets they used are relatively small due to the intensive manual work, e.g. [13] used two datasets each containing 92 Caucasian female face images [17], used a dataset containing 210 images for each gender, and the sizes of datasets in [14], [15] are 91 and 92, respectively.

Two kinds of approaches can be used to analyze the relationship between facial features and facial attractiveness, namely deductive and inductive approaches. [8], [24] used deductive approach. They generated stimuli according to empirical hypotheses, and then examined the effectiveness of the hypotheses. Some recent studies [13], [14], [15], [17] used inductive approach. They collected human ratings of the attractiveness of the faces in their databases, and then established the relationship between facial features and facial attractiveness based on the human ratings by using machine learning methods. Using the obtained relationship, they predicted the attractiveness of a given face and compared the prediction results with human ratings of the attractiveness of the face.

Several facial beauty hypotheses have been raised in the past studies, e.g. symmetrical faces [19], [20], [21], average faces [11], [22], or faces with exaggerated secondary sexual characteristics [23], [24] are found attractive. Among these hypotheses, the most influential and most investigated one is the averageness hypothesis, which assumes that a beautiful face is simply one that is close to an average of all faces, notwithstanding the particular features of those faces. In support of this some researchers have shown that composite faces, digitally blended and averaged, are regarded as more attractive than most of the faces used to create them [11], [18], [22]. One objection to the averageness hypothesis, however, is that the attractiveness of average composite faces may be due to other factors co-varying with the averaging operation such as the smoothed skin texture and the improved symmetry of the average composite faces [25]. Also challenging was the finding of Perrett et al. [8] that composites of beautiful people were rated more appealing than composites from the larger, random population. In order to investigate the effect of individual factor on facial beauty, some researchers manipulated the averageness on shape and texture separately [18], [21], [24]. In their experiments, it was reported that individual faces warped into average face shapes were rated as more attractive than the original, and that decreasing averageness by moving the faces away from average face shapes decreased attractiveness. Some other researchers dissociated symmetry and averageness in facial beauty perception experiments and showed that symmetry does contribute positively to facial beauty, but it cannot solely explain the attractiveness of average faces [24].

Despite the increasing public and professional interest and the extensive research in the field, the published studies have not yet reached consensus on those facial beauty hypotheses or on the characteristics that make human faces attractive. Moreover, there are some limitations in previous studies on facial beauty. First, the difference between face shapes was not quantified and it was consequently hard to quantitatively assess the impact of deviation from average face shapes on facial beauty. Second, they ignored the transformation, e.g. scaling and rotation, occurring to the faces during generation of face images. Such transformation can however greatly affect the calculated distances between face shapes, and thus impair the reliability of the conclusions. They did not normalize the physical sizes of faces when evaluating their attractiveness, yet as common observations in our daily life tell us, the distance between a small-sized face to a large-sized face could be very large, but the facial beauty perception of them could be similar. Third, most of them require intensive manual work, e.g. marking landmarks. Forth, most existing studies are based on human ratings of the attractiveness of faces. The quality of the collected human ratings severely affects the results. According to our experiments, if an absolute attractiveness score in a certain scale (e.g. 10-point scale [17] or 7-point scale [8], [13], [14]) is required to assign to a face, large variation would be observed among different people. Unfortunately, only the averaging human ratings were used as ground truth in [13], [14], [15], [17], but the variance of human ratings was not reported or analyzed. Moreover, when collecting the attractiveness scores of faces, they did not isolate the facial geometric features from other features. As a consequence, the collected attractiveness scores depend not just on geometric features, but also on other features like textures. It is thus problematic to study the relationship between geometric features and facial beauty based on such kind of attractiveness scores.

The goal of this preliminary study is to quantitatively analyze the effect of facial geometric features on the attractiveness of faces. In order to conquer the above-mentioned limitations of existing studies, we consider human faces in a feature space, namely human face shape space, which is defined by the facial geometric features and is a subspace of the unit hypersphere. Every human face after normalization is represented by a point in the space. The distance between two face shapes in the human face shape space is measured in terms of the angle between them, and an effective algorithm is proposed to calculate it such that the measured distance is invariant to similarity transformation (i.e. translation, rotation, and scaling). Based on the established human face shape space and the distance measurement and by using advanced automatic image processing techniques, we are enabled to quantitatively evaluate the human face shapes and their contribution to the human facial beauty perception without interference of non-geometric features. Moreover, we employ a deductive approach in this study, i.e. human ratings are only used for evaluation experiments (instead of scoring each individual faces, participants are requested to choose out the most beautiful face from a set of faces which differ only in their geometric features).

The rest of this paper is organized as follows. In Section 2, we introduce in detail the human face shape space, the shape distance measurement, the average face shapes, and associated algorithms, which compose the basis of the quantitative analysis. In Section 3, we then introduce the image processing techniques and the face stimuli we employed in the experiments, as well as the design of the perception experiments in this paper. In Section 4, we report and analyze the results we have obtained and the main findings of the study will be also presented there, including (i) the human face shapes are condensed in a very small region of the feature space, (ii) the female and male average face shapes are very similar, and (iii) faces generally become more beautiful as their geometric features get obviously closer to the average face shapes, while it cannot effectively improve the attractiveness of a face which is already relatively close to the average face shape by further deforming it toward the average face shape. The paper is finally concluded in Section 5 with discussion on future work.