Abstract
This paper presents a novel method to organize a collection of images into a hierarchy of clusters based on image semantics. Given a group of raw images with no metadata as input, our method describes the semantics of each image with a bag-of-semantics model (i.e., a set of meaningful descriptors), which is derived from the image’s Object Relation Network (Chen et al. in Proceedings of the 21st International Conference on World Wide Web, 2012)—an expressive graph model representing rich semantics for image objects and their relations. We adopt the class hierarchies in a guide ontology as different levels of lenses to view the bag-of-semantics models. Image clusters are automatically extracted by grouping images with the same bag-of-semantics viewed through a certain lens. With a series of coarse-to-fine lenses, images are clustered in a top-down hierarchical manner. In addition, given that users can have different perspectives regarding how images should be clustered, our method allows each user to control the clustering process while browsing, and thus dynamically adjusts the clustering result according to the user’s preferences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bi, J., Chen, Y., Wang, J.Z.: A sparse support vector machine approach to region-based image categorization. In: CVPR (2005)
Biswas, A., Jacobs, D.: Active image clustering: seeking constraints from humans to complement algorithms. In: CVPR (2012)
Bosch, A., Zisserman, A., Muñoz, X.: Scene classification via plsa. In: ECCV (2006)
Cai, D., He, X., Li, Z., Ma, W.Y., Wen, J.R.: Hierarchical clustering of www image search results using visual, textual and link information. In: ACM Multimedia (2004)
Chen, N., Prasanna, V.: Semantic image clustering using object relation network. In: Proceedings of the First International Conference on Computational Visual Media (2012)
Chen, N., Zhou, Q.Y., Prasanna, V.: Understanding web images by object relation network. In: Proceedings of the 21st International Conference on World Wide Web (2012)
Chen, T., Cheng, M.M., Tan, P., Shamir, A., Hu, S.M.: Sketch2photo: Internet image montage. In: ACM SIGGRAPH Asia 2009 Papers (2009)
Chen, Y., Wang, J.Z.: Image categorization by learning and reasoning with regions. J. Mach. Learn. Res. (2004)
Chen, Y., Wang, J.Z., Krovetz, R., Clue: Cluster-based retrieval of images by unsupervised learning. IEEE Trans. Image Process. (2003)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: CVPR (2009)
Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2011 (VOC2011) results. http://www.pascal-network.org/challenges/VOC/voc2011/workshop/index.html
Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9) (2010)
Gao, B., Liu, T.Y., Qin, T., Zheng, X., Cheng, Q.S., Ma, W.Y.: Web image clustering by consistent utilization of visual features and surrounding texts. In: ACM Multimedia (2005)
Gemert, J.C., Geusebroek, J.M., Veenman, C.J., Smeulders, A.W.: Kernel codebooks for scene categorization. In: ECCV (2008)
Gordon, S., Greenspan, H., Goldberger, J.: Applying the information bottleneck principle to unsupervised clustering of discrete and continuous image representations. In: ICCV (2003)
He, X., Zemel, R.S., Carreira-Perpinan, M.A.: Multiscale conditional random fields for image labeling. In: CVPR (2004)
Jing, F., Wang, C., Yao, Y., Deng, K., Zhang, L., Ma, W.Y.: Igroup: web image search results clustering. In: ACM Multimedia (2006)
Ladicky, L., Sturgess, P., Alahari, K., Russell, C., Torr, P.H.: What, where and how many? Combining object detectors and crfs. In: ECCV (2010)
Li, F.F., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: CVPR (2005)
Li, L.J., Socher, R., Fei-Fei, L.: Towards total scene understanding: classification, annotation and segmentation in an automatic framework. In: CVPR (2009)
Liu, Y., Chen, X., Zhang, C., Sprague, A.: Semantic clustering for region-based image retrieval. J. Vis. Comun. Image Represent. (2009)
Nguyen, B.P., Tay, W.L., Chui, C.K., Ong, S.H.: A clustering-based system to automate transfer function design for medical image visualization. Vis. Comput. (2012)
Nwogu, I., Govindaraju, V., Brown, C.: Syntactic image parsing using ontology and semantic descriptions. In: CVPR (2010)
Rodden, K., Basalaj, W., Sinclair, D., Wood, K.: Does organisation by similarity assist image browsing? In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (2001)
Torralba, A., Murphy, K., Freeman, W.T.: Using the forest to see the trees: exploiting context for visual object detection and localization. In: Commun. ACM (2010)
Wang, X.J., Ma, W.Y., Zhang, L., Li, X.: Iteratively clustering web images based on link and attribute reinforcements. In: ACM Multimedia (2005)
Xu, K., Li, Y., Ju, T., Hu, S.M., Liu, T.Q.: Efficient affinity-based edit propagation using k-d tree. In: ACM SIGGRAPH Asia 2009 Papers (2009)
Zheng, X., Cai, D., He, X., Ma, W.Y., Lin, X.: Locality preserving clustering for image database. In: ACM Multimedia (2004)
Acknowledgements
This work is supported by Chevron Corp. under the joint project, Center for Interactive Smart Oilfield Technologies (CiSoft), at the University of Southern California.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, N., Prasanna, V.K. A bag-of-semantics model for image clustering. Vis Comput 29, 1221–1229 (2013). https://doi.org/10.1007/s00371-013-0785-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-013-0785-5