Auto Accessory Segmentation and Interactive Try-on System

Abstract

The convenience and diversity of online shopping makes many consumers willing to buy apparel or accessories on the web. In order to make products more attractive to users, many virtual try-on systems are developed for e-commerce applications. This paper proposes an interactive virtual try-on system combined with automatic accessory segmentation. Our system automatically retrieves the hat from images and store them in the try-on system to provide users with subsequent selection. When a user selects the hat that he or she wants to try on, the hat is placed on the proper position of the user in the image. In the stage of accessories segmentation, we perform background elimination and super-pixel segmentation. According to the color information on the hat image, the feature vector generated by the color histogram is used to select super-pixels that belong to the accessories. In the stage of try-on system, we use Kinect, which provides skeleton information, to track the user’s face and gestures. When a user selects the hat, the proposed system reads the corresponding hat information and places the hat in the appropriate location based on the results of the face tracking. The proposed try-on system can reach 30 fps real-time speed in a personal computer.