Sparse representations based attribute learning for flower classification

Abstract

Classification for flowers is a very difficult task. Traditional methods need to built a classifier for each flower category, and obtain large number of flower samples to train these classifiers. In practice, many different types of flowers make the job become very difficult and boring. In this work, we present an attribute based approach for flowers recognition. Particularly, instead of training for a specific category of flowers directly based on manually designed features such as SIFT and HoG, we extract a series of visual attributes from a given set of flower images and generalize these to new images with possibly unknown flowers. A recently proposed sparse representations classification scheme is employed to predict the attributes of a given flower image from any category. In addition, we use a genetic algorithm to find the most discriminative attributes among others for better performance during the stage of flower classification. The effectiveness of the proposed method is validated on a publicly available flower classification database with promising results.

Introduction

The task of classification for flowers is very difficult. There are large variations in scale and viewpoint in typical flower images, partial occlusions, illumination, multiple instances, etc. Perhaps the biggest challenge comes from the intra-class and the inter-class variability. For example, some images from different classes are with a smaller variation than from a class itself, and some minute differences determine their different classification [1], [2], [3], [4], [5], [6], [7]. In addition, as a plant with continuous growth and a non-rigid object, flowers can be deformed in many ways, so in the intra-class there is also a big change. Recent machine learning technique has demonstrated its power in classifying flowers. For example, one can consider kernel SVM [8] or boosting [9]. The performance of the state of the art method is obtained with the use of multiple features [10] and a multiple kernel classifier: each kernel is designed for different features (e.g. colour, texture), and an additional kernel is designed for the weighted combination of these feature kernels.

Recently, along with the emergence of camera-equipped mobile phones, new opportunity and challenge have generated in the computer vision field. One of these challenges is that in resource constrained environments, available memory, bandwidth and processing power become restricting factors in image classification [11]. For example, with a camera-equipped smart phone, a user captures a flower image and wants to learn more knowledge about the flower. In a traditional image identification system, it would either extract some type of features of the flower image and transmit it to a server or transmit the whole flower image to the server directly. Then, for recognizing the category of the flower image and feeding back the depicted information of the flower, the server needs to perform a series of classification tests.

Regarding the feasibility of flowers classifying, there are two important issues. The first one is that there should be enough labeled examples for training the classifier. Then we can use the classifier to identify the possible category of new test examples having the same distribution with the training examples. However, due to the large number of the possible image classes, collecting a large amount of examples for training may not be feasible even though the number of classes is not a large one [12]. In addition, training classifiers of a real-time application in the server may also be impractical, because along with the increase of visitors, the performance of system response will drop quickly.

The second one which leads to the failure of system design is the limited power availability, bandwidth and processing capabilities of mobile systems. For example, the available battery power will quickly drain when transmitting raw images. Besides, because the transmitted information load also increase congestion on the server and the network, the user satisfaction will be directly affected.

To cope with the challenges mentioned above, attribute learning proposed recently is attempted to learn attributes instead of traditional classes of image. In the camera-equipped smart phone system shown in Fig. 1, it could recognize the attributes e.g. “red” or “green” for the specific classes׳ classification. The processing pipeline includes two stages: the one is the attribute vector creating process which is accomplished by the mobile facility, and the other one is final classification taken place on the server. The classification stage is learned off-line from textual description, which gives consideration to the mapping of attributes to classes.

There are many advantages in employing attributes to be a bridge between image examples and classes. The greatest benefit from the use of attributes is that classifiers can be trained and tested using only text information without direct image features. This means that instead of using label sample as metadata, and in order to automatically identify test examples, large knowledge databases can be employed for extracting information. Besides, textual information is easier to transmit, process and store than the image information. Compared to raw images, the text of attributes with smaller dimensions can be easier employed to reduce the query time for a huge retrieval system. Considering that the visits to the server is very large at the same time, the advantage mentioned above becomes very vital for a system like the one shown in Fig. 1.

In recent years, as a kind of high-level image feature, the semantic attribute has gained more and more attention in the field of computer vision. Attributes׳ learning has been applied widespread in classifying objects [13] or images [14]. Farhadi et al. [15] first put forward a set of visual semantic attributes to describe objects. Later, Kumar et al. [16] proposed a novel method to predict the visual attribute vector through the related attribute classifiers, and using these attribute vectors to represent faces. Vogel and Schiele [17] employed visual attributes to express the semantics of the outdoor scenes. Vaquero et al. proposed an attribute based people search method [18]. Attributes learning also has many potential applications in transfer learning [19], multi-label learning [20], multi-instance learning [21], video annotation [22] and image retrieval [23]. Despite that the semantic attributes have been used in many kinds of image classifications, to our knowledge there is no system that identifies flowers using their attributes.

As far as the power availability and the communication bandwidth of mobile systems are concerned, when the raw images or image descriptors are limitedly transferred, attribute expression can still be transmitted. In addition, instead of integrating the whole system of images recognition, only two processing stages are needed in the mobile system. The first one is the feature extraction, and the second is the attribute prediction. In recent years, feature extraction and image classification have been applied in smart phones and other mobile devices [24], [25], [26] and have got very exciting results.

Attribute based identification as a new way for image classification can be used to deal with the flowers classifying based on camera-equipped mobile phones. However, for the existence of a large number of available attributes, collecting samples to train the attribute classifiers is a tedious task. In addition, the traditional attribute based classification framework assumes independence among attributes, which cannot be ensured by the attributes annotated by human. In this paper we propose an integrative framework that enables us to predict attribute automatically and to estimate the prior attribute–class probability relationship matrix. We use some image samples to compose the dictionary of the attribute classifier and employ the recently proposed sparse representations classification scheme to predict other samples attribute. Besides, in order to find the most discriminative attributes, a genetic algorithm is employed to reduce attributes for the flower classification. The benefits of the proposed extensions are validated through the attribute-to-class mapping experimental results.