skip to main content
research-article

Deep Scalable Supervised Quantization by Self-Organizing Map

Published: 20 August 2019 Publication History

Abstract

Approximate Nearest Neighbor (ANN) search is an important research topic in multimedia and computer vision fields. In this article, we propose a new deep supervised quantization method by Self-Organizing Map to address this problem. Our method integrates the Convolutional Neural Networks and Self-Organizing Map into a unified deep architecture. The overall training objective optimizes supervised quantization loss as well as classification loss. With the supervised quantization objective, we minimize the differences on the maps between similar image pairs and maximize the differences on the maps between dissimilar image pairs. By optimization, the deep architecture can simultaneously extract deep features and quantize the features into suitable nodes in self-organizing map. To make the proposed deep supervised quantization method scalable for large datasets, instead of constructing a larger self-organizing map, we propose to divide the input space into several subspaces and construct self-organizing map in each subspace. The self-organizing maps in all the subspaces implicitly construct a large self-organizing map, which costs less memory and training time than directly constructing a self-organizing map with equal size. The experiments on several public standard datasets prove the superiority of our approaches over the existing ANN search methods. Besides, as a by-product, our deep architecture can be directly applied to visualization with little modification, and promising performance is demonstrated in the experiments.

References

[1]
Yue Cao, Bin Liu, Mingsheng Long, Jianmin Wang, and MOE Kliss. 2018. HashGAN: Deep learning to hash with pair conditional wasserstein GAN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1287--1296.
[2]
Yue Cao, Mingsheng Long, Bin Liu, Jianmin Wang, and MOE Kliss. 2018. Deep cauchy hashing for hamming space retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1229--1237.
[3]
Yue Cao, Mingsheng Long, Jianmin Wang, Han Zhu, and Qingfu Wen. 2016. Deep quantization network for efficient image retrieval. In Association for the Advancement of Artificial Intelligence. 3457--3463.
[4]
Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the devil in the details: Delving deep into convolutional nets. In Proceedings of the Association for the Advancement of Artificial Intelligence (2014).
[5]
Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. 2009. NUS-WIDE: A real-world web image database from National University of Singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval. 48.
[6]
Thanh-Toan Do, Dang-Khoa Le Tan, Trung T. Pham, and Ngai-Man Cheung. 2017. Simultaneous feature aggregating and hashing for large-scale image search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6618--6627.
[7]
Yueqi Duan, Jiwen Lu, Ziwei Wang, Jianjiang Feng, and Jie Zhou. 2017. Learning deep binary descriptor with multi-quantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1183--1192.
[8]
Kamran Ghasedi Dizaji, Feng Zheng, Najmeh Sadoughi, Yanhua Yang, Cheng Deng, and Heng Huang. 2018. Unsupervised deep generative adversarial hashing network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3664--3673.
[9]
Yunchao Gong, Svetlana Lazebnik, Albert Gordo, and Florent Perronnin. 2013. Iterative quantization: A procrustean approach to learning binary codes for large-scale image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 35, 12 (2013), 2916--2929.
[10]
Yen-Chang Hsu and Zsolt Kira. 2015. Neural network-based clustering using pairwise constraints. arXiv preprint arXiv:1511.06321 (2015).
[11]
Yen-Chang Hsu, Zhaoyang Lv, and Zsolt Kira. 2016. Deep image category discovery using a transferred similarity function. arXiv preprint arXiv:1612.01253 (2016).
[12]
Mengqiu Hu, Yang Yang, Fumin Shen, Ning Xie, Richang Hong, and Heng Tao Shen. 2019. Collective reconstructive embeddings for cross-modal hashing. IEEE Trans. Image Process. 28, 6 (2019), 2770--2784.
[13]
Mengqiu Hu, Yang Yang, Fumin Shen, Ning Xie, and Heng Tao Shen. 2018. Hashing with angular reconstructive embeddings. IEEE Trans. Image Process. 27, 2 (2018), 545--555.
[14]
Himalaya Jain, Joaquin Zepeda, Patrick Pérez, and Rémi Gribonval. 2017. SUBIC: A supervised, structured binary code for image search. In Proceedings of the IEEE International Conference on Computer Vision, Vol. 1. 3.
[15]
Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2011. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1 (2011), 117--128.
[16]
Qing-Yuan Jiang and Wu-Jun Li. 2018. Asymmetric deep supervised hashing. In Proceedings of the Association for the Advancement of Artificial Intelligence (2018).
[17]
Yu-Gang Jiang, Jun Wang, Xiangyang Xue, and Shih-Fu Chang. 2013. Query-adaptive image search with hash codes. IEEE Trans. Multimedia 15, 2 (2013), 442--453.
[18]
Teuvo Kohonen. 1982. Self-organized formation of topologically correct feature maps. Biol. Cybernet. 43, 1 (1982), 59--69.
[19]
Teuvo Kohonen and Timo Honkela. 2007. Kohonen network. Scholarpedia 2, 1 (2007), 1568.
[20]
Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. (2009). Technical Report.
[21]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 1097--1105.
[22]
Hanjiang Lai, Yan Pan, Ye Liu, and Shuicheng Yan. 2015. Simultaneous feature learning and hash coding with deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3270--3278.
[23]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.
[24]
Qi Li, Zhenan Sun, Ran He, and Tieniu Tan. 2017. Deep supervised discrete hashing. In Advances in Neural Information Processing Systems. 2482--2491.
[25]
Wu-Jun Li, Sheng Wang, and Wang-Cheng Kang. 2016. Feature learning based deep supervised hashing with pairwise labels. In Proceedings of the International Joint Conference on Artificial Intelligence.
[26]
Renjie Liao, Alex Schwing, Richard Zemel, and Raquel Urtasun. 2016. Learning deep parsimonious representations. In Advances in Neural Information Processing Systems. 5076--5084.
[27]
Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen. 2016. Deep supervised hashing for fast image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2064--2072.
[28]
Zhen Liu, Houqiang Li, Wengang Zhou, Ruizhen Zhao, and Qi Tian. 2014. Contextual hashing for large-scale image search. IEEE Trans. Image Process. 23, 4 (2014), 1606--1614.
[29]
Jiwen Lu, Venice Erin Liong, and Jie Zhou. 2017. Deep hashing for scalable image search. IEEE Trans. Image Process. 26, 5 (2017), 2352--2367.
[30]
Yadan Luo, Yang Yang, Fumin Shen, Zi Huang, Pan Zhou, and Heng Tao Shen. 2018. Robust discrete code modeling for supervised hashing. Pattern Recogn. 75, 1 (2018), 128--135.
[31]
Mohammad Norouzi and David J. Fleet. 2013. Cartesian k-means. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3017--3024.
[32]
Pierre Sermanet, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun. 2013. Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013).
[33]
Fumin Shen, Xin Gao, Li Liu, Yang Yang, and Heng Tao Shen. 2017. Deep asymmetric pairwise hashing. In Proceedings of the ACM International Conference on Multimedia. ACM, 1522--1530.
[34]
Fumin Shen, Yan Xu, Li Liu, Yang Yang, Zi Huang, and Heng Tao Shen. 2018. Unsupervised deep hashing with similarity-adaptive and discrete optimization. IEEE Trans. Pattern Anal. Mach. Intell. 40, 12 (2018), 3034--3044.
[35]
Andrea Vedaldi and Karel Lenc. 2015. Matconvnet: Convolutional neural networks for matlab. In Proceedings of the ACM International Conference on Multimedia. ACM, New York, NY, 689--692.
[36]
Daixin Wang, Peng Cui, Mingdong Ou, and Wenwu Zhu. 2015. Learning compact hash codes for multimodal representations using orthogonal deep structure. IEEE Trans. Multimedia 17, 9 (2015), 1404--1416.
[37]
Jianfeng Wang, Jingdong Wang, Nenghai Yu, and Shipeng Li. 2013. Order preserving hashing for approximate nearest neighbor search. In Proceedings of the ACM International Conference on Multimedia. 133--142.
[38]
Min Wang, Wengang Zhou, Qi Tian, and Houqiang Li. 2018. A general framework for linear distance preserving hashing. IEEE Trans. Image Process. 27, 2 (2018), 907--922.
[39]
Min Wang, Wengang Zhou, Qi Tian, Junfu Pu, and Houqiang Li. 2017. Deep supervised quantization by self-organizing map. In Proceedings of the ACM International Conference on Multimedia. ACM, 1707--1715.
[40]
Min Wang, Wengang Zhou, Qi Tian, Zhengjun Zha, and Houqiang Li. 2016. Linear distance preserving pseudo-supervised and unsupervised hashing. In Proceedings of the ACM International Conference on Multimedia. ACM, 1257--1266.
[41]
Xiaojuan Wang, Ting Zhang, Guo-Jun Qi, Jinhui Tang, and Jingdong Wang. 2016. Supervised quantization for similarity search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018--2026.
[42]
Yair Weiss, Antonio Torralba, and Rob Fergus. 2009. Spectral hashing. In Advances in Neural Information Processing Systems. 1753--1760.
[43]
Rongkai Xia, Yan Pan, Hanjiang Lai, Cong Liu, and Shuicheng Yan. 2014. Supervised hashing for image retrieval via image representation learning. In Association for the Advancement of Artificial Intelligence, Vol. 1. 2.
[44]
Litao Yu, Yongsheng Gao, and Jun Zhou. 2018. Generative adversarial product quantisation. In Proceedings of the ACM Conference on Multimedia. ACM, 861--869.
[45]
Tan Yu, Junsong Yuan, Chen Fang, and Hailin Jin. 2018. Product quantization network for fast image retrieval. In Proceedings of the European Conference on Computer Vision. 186--201.
[46]
Haofeng Zhang, Li Liu, Yang Long, and Ling Shao. 2018. Unsupervised deep hashing with pseudo labels for scalable image retrieval. IEEE Trans. Image Process. 27, 4 (2018), 1626--1638.
[47]
Lei Zhang, Yongdong Zhang, Jinhui Tang, Xiaoguang Gu, Jintao Li, and Qi Tian. 2013. Topology preserving hashing for similarity search. In Proceedings of the ACM International Conference on Multimedia. 123--132.
[48]
Ruimao Zhang, Liang Lin, Rui Zhang, Wangmeng Zuo, and Lei Zhang. 2015. Bit-scalable deep hashing with regularized similarity learning for image retrieval and person re-identification. IEEE Trans. Image Process. 24, 12 (2015), 4766--4779.
[49]
Ting Zhang, Chao Du, and Jingdong Wang. 2014. Composite quantization for approximate nearest neighbor search. In Proceedings of the International Conference on Machine Learning. 838--846.
[50]
Fang Zhao, Yongzhen Huang, Liang Wang, and Tieniu Tan. 2015. Deep semantic ranking based hashing for multi-label image retrieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1556--1564.
[51]
Wengang Zhou, Houqiang Li, Richang Hong, Yijuan Lu, and Qi Tian. 2015. BSIFT: Toward data-independent codebook for large scale image search. IEEE Trans. Image Process. 24, 3 (2015), 967--979.
[52]
Wengang Zhou, Yijuan Lu, Houqiang Li, and Qi Tian. 2012. Scalar quantization for large scale image search. In Proceedings of the ACM International Conference on Multimedia. 169--178.
[53]
Wengang Zhou, Ming Yang, Houqiang Li, Xiaoyu Wang, Yuanqing Lin, and Qi Tian. 2014. Towards codebook-free: Scalable cascaded hashing for mobile image search. IEEE Trans. Multimedia 16, 3 (2014), 601--611.
[54]
Wengang Zhou, Ming Yang, Xiaoyu Wang, Houqiang Li, Yuanqing Lin, and Qi Tian. 2016. Scalable feature matching by dual cascaded scalar quantization for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 38, 1 (2016), 159--171.
[55]
Hao Zhu and Shenghua Gao. 2017. Locality-constrained deep supervised hashing for image retrieval. In Proceedings of the International Joint Conference on Artificial Intelligence. 3567--3573.
[56]
Weijia Zhu, Wenpeng Ding, Jizheng Xu, Yunhui Shi, and Baocai Yin. 2015. Hash-based block matching for screen content coding. IEEE Trans. Multimedia 17, 7 (2015), 935--944.

Cited By

View all
  • (2023)Weakly Supervised Hashing with Reconstructive Cross-modal AttentionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3589185Online publication date: 8-Apr-2023
  • (2021)Machine Learning Modeling of Water Use Patterns in Small Disadvantaged CommunitiesWater10.3390/w1316231213:16(2312)Online publication date: 23-Aug-2021
  • (2020)Deep Convolutional Self-Organizing Map Network for Robust Handwritten Digit RecognitionIEEE Access10.1109/ACCESS.2020.30008298(107035-107045)Online publication date: 2020

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 15, Issue 3
August 2019
331 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/3352586
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2019
Accepted: 01 April 2019
Revised: 01 March 2019
Received: 01 October 2018
Published in TOMM Volume 15, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Approximate nearest neighbor search
  2. self-organizing map
  3. supervised quantization

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Young Elite Scientists Sponsorship Program By CAST
  • NSFC

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)21
  • Downloads (Last 6 weeks)4
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Weakly Supervised Hashing with Reconstructive Cross-modal AttentionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/3589185Online publication date: 8-Apr-2023
  • (2021)Machine Learning Modeling of Water Use Patterns in Small Disadvantaged CommunitiesWater10.3390/w1316231213:16(2312)Online publication date: 23-Aug-2021
  • (2020)Deep Convolutional Self-Organizing Map Network for Robust Handwritten Digit RecognitionIEEE Access10.1109/ACCESS.2020.30008298(107035-107045)Online publication date: 2020

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media