Abstract
This paper presents a new framework, ClothingOut, which utilizes generative adversarial network (GAN) to generate tiled clothing images automatically. Specifically, we design a novel category-supervised GAN model by learning transformation rules between clothes on wearers and clothes that are tiled. Our method features in adding category attribute to a traditional GAN model. For model training, we built a large-scale dataset containing over 20,000 pairs of wearer images and their corresponding tiled clothing images. The learned model can be straightforwardly applied to video advertising and cross-scenario clothing image retrieval. We evaluated our generated images which can be regarded as the segmentation from the wearer images from two aspects: authenticity and retrieval performance. Experimental results demonstrate the effectiveness of our method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhang H, Cao X, Ho JKL, Chow TWS (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inf 13(2):520–531
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets, In: Advances in neural information processing systems, pp 2672–2680
Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks,’ arXiv:1511.06434
Mirza M, Osindero S (2014) Conditional generative adversarial nets, arXiv:1411.1784v1
Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: CVPR
Girshick R, Donahue J, Darrell T, Malik J (2014) UC Berkeley and ICSI, Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR
Uijlings JRR, van de Sande KEA, Gevers T, Smeulders AWM (2013) Selective search for object recognition. Int J Comput Vis 104(2):154–171
Ross G (2015) Fast R-CNN. In: CVPR
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: CVPR
Redmon J, Farhadi A (2016) YOLO9000: better, faster, stronger, arXiv:1612.08242v1
Wan J, Wang D, Hoi SCH, Wu P, Zhu J, Zhang Y, Li J (2014) Deep learning for content-based image retrieval: a comprehensive study. In: MM ’14 proceedings of the 22nd ACM international conference on multimedia, pp 157–166
Veit A, Kovacs B, Bell S, McAuley J, Bala K, Belongie S (2015) Learning visual clothing style with heterogeneous dyadic co-occurrences. In: ICCV
Wang X, Shi Y, Kitani KM (2016) Deep supervised hashing with triplet labels. In: Asian conference on computer vision, pp 70–84
Liu Z, Luo P, Qiu S, Wang X, Tang X (2016) DeepFashion: powering Robust clothes recognition and retrieval with Rich annotations. In: CVPR
Lin K, Yang H-F, Hsiao J-H, Cheni C-S (2015) Deep learning of binary hash codes for fast image retrieval. In: CVPR
Zhu S, Fidler S, Urtasun R, Lin D, Loy CC (2017) Be your own prada: fashion synthesis with structural coherence. In: ICCV
Wan X, Gupta A (2016) Generative image modeling using style and structure adversarial networks. In: European conference on computer vision, pp 318–335
Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp 234–241
Zeiler Matthew D, Fergus Rob (2014) Visualizing and understanding convolutional networks. In: ECCV 2014: computer vision, pp 818–833
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. In: Computer vision and pattern recognition, arXiv:1409.1556v6
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) SSD: single shot multibox detector. In: ECCV 2016: computer vision, pp 21–37
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: CVPR
Zhang H, Chow TWS, Wu QMJ (2016) Organizing books and authors by multilayer SOM sign in or purchase. IEEE Trans Neural Netw Learn Syst 27(12):2537–2550
Zhang H, Xu T, Li H, Zhang S, Huang X, Wang X, Metaxas D (2016) StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks, arXiv:1612.03242
Acknowledgements
This work was supported in part by the National Key R&D Program of China under Grant No. 2018YFB1003800, the Natural Science Foundation of China under Grant No. 61572156, and the Shenzhen Science and Technology Program under Grant Nos. JCYJ20170413105929681 and JCYJ20170811161545863.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that no conflict of interest.
Rights and permissions
About this article
Cite this article
Zhang, H., Sun, Y., Liu, L. et al. ClothingOut: a category-supervised GAN model for clothing segmentation and retrieval. Neural Comput & Applic 32, 4519–4530 (2020). https://doi.org/10.1007/s00521-018-3691-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-018-3691-y