Abstract
Clothing, as a kind of beauty-enhancing product, plays an important role in people’s daily life. People want to look good by dressing properly. Nevertheless, not everyone is good at clothing matching and thus is able to make aesthetic outfits. Fortunately, certain fashion-oriented online community (e.g., Polyvore) allows fashion experts to share their outfit compositions to the public. Each outfit composition there usually consists of several complementary items (e.g., tops, bottoms and shoes), where both the visual image and textual title are available for each item. In this work, we aim to take fully advantage of such rich fashion data to decode the secret of clothing matching. Essentially, we propose a method (CMVT) to comprehensively measure the compatibility among fashion items by integrating the multi-modal data of items. Extensive experiments have been conducted on a real-world dataset to evaluate the effectiveness of the proposed model.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Bottou, L.: Stochastic gradient learning in neural networks. Proc. Neuro-Nımes 91(8) (1991)
Chen, J., Song, X., Nie, L., Wang, X., Zhang, H., Chua, T.-S.: Micro tells macro: predicting the popularity of micro-videos via a transductive model. In: Proceedings of the ACM International Conference on Multimedia, pp. 898–907. ACM (2016)
Gan, T., Wong, Y., Zhang, D., Kankanhalli, M.S.: Temporal encoded f-formation system for social interaction detection. In: Proceedings of the ACM International Conference on Multimedia, pp. 937–946. ACM (2013)
Gao, Y., Wang, M., Zha, Z.-J., Shen, J., Li, X., Xindong, W.: Visual-textual joint relevance learning for tag-based social image search. IEEE Trans. Image Process. 22(1), 363–376 (2013)
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., Chua, T.-S.: Neural collaborative filtering. In: Proceedings of the ACM International Conference on World Wide Web, pp. 173–182. ACM (2017)
He, X., Zhang, H., Kan, M.-Y., Chua, T.-S.: Fast matrix factorization for online recommendation with implicit feedback. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 549–558. ACM (2016)
Hong, R., Zhenzhen, H., Wang, R., Wang, M., Tao, D.: Multi-view object retrieval via multi-scale topic models. IEEE Trans. Image Process. 25(12), 5814–5827 (2016)
Hong, R., Yang, Y., Wang, M., Hua, X.-S.: Learning visual semantic relationships for efficient visual retrieval. IEEE Trans. Big Data 1(4), 152–161 (2015)
Hong, R., Zhang, L., Zhang, C., Zimmermann, R.: Flickr circles: aesthetic tendency discovery by multi-view regularized topic modeling. IEEE Trans. Multimed. 18(8), 1555–1567 (2016)
Ji, R., Xie, X., Yao, H., Ma, W.-Y.: Mining city landmarks from blogs by graph modeling. In: Proceedings of the ACM International Conference on Multimedia, pp. 105–114. ACM (2009)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
McAuley, J., Targett, C., Shi, Q., Van Den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52. ACM (2015)
Nie, L., Wang, M., Zha, Z.-J., Chua, T.-S.: Oracle in image search: a content-based approach to performance prediction. ACM Trans. Inf. Syst. 30, 13:1–13:23 (2012)
Nie, L., Wang, M., Zha, Z., Li, G., Chua, T.-S.: Multimedia answering: enriching text QA with media information. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 695–704. ACM (2011)
Nie, L., Yan, S., Wang, M., Hong, R., Chua, T.-S.: Harvesting visual concepts for image search with complex queries. In: Proceedings of the ACM International Conference on Multimedia, pp. 59–68. ACM (2012)
Jagadeesh, V., Piramuthu, R., Bhardwaj, A., Di, W., Sundaresan, N.: Large scale visual recommendations from street fashion images. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1925–1934. ACM (2014)
Song, X., Feng, F., Liu, J., Li, Z., Nie, L., Ma, J.: Neurostylist: neural compatibility modeling for clothing matching. In: Proceedings of the ACM International Conference on Multimedia. ACM (2017)
Song, X., Ming, Z.-Y., Nie, L., Zhao, Y.-L., Chua, T.-S.: Volunteerism tendency prediction via harvesting multiple social networks. ACM Trans. Inf. Syst. 34(2), 10 (2016)
Song, X., Nie, L., Zhang, L., Akbari, M., Chua, T.-S.: Multiple social network learning and its application in volunteerism tendency prediction. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 213–222. ACM (2015)
Voorhees, E.M., et al.: The TREC-8 question answering track report. In: TREC, vol. 99, pp. 77–82 (1999)
Iwata, T., Wanatabe, S., Sawada, H.: Fashion coordinates recommender system using photographs from fashion magazines. In: Proceedings of the International Joint Conference on Artificial Intelligence, vol. 22 (2011)
Wang, X., He, X., Nie, L., Chua, T.-S.: Item silk road: recommending items from information domains to social users. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM (2017)
Zhang, H., Shang, X., Yang, W., Xu, H., Luan, H., Chua, T.-S.: Online collaborative learning for open-vocabulary visual classifiers. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2809–2817. IEEE (2016)
Acknowledgments
The work is supported by the National Natural Science Foundation of China under Grant No.: 61702300.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, Z., Song, X., Gan, T., Chen, Z., Zhu, X. (2018). Clothing Matching Based on Multi-modal Data. In: Huet, B., Nie, L., Hong, R. (eds) Internet Multimedia Computing and Service. ICIMCS 2017. Communications in Computer and Information Science, vol 819. Springer, Singapore. https://doi.org/10.1007/978-981-10-8530-7_7
Download citation
DOI: https://doi.org/10.1007/978-981-10-8530-7_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-8529-1
Online ISBN: 978-981-10-8530-7
eBook Packages: Computer ScienceComputer Science (R0)