Skip to main content
SpringerLink
Log in

Semi-supervised image clustering with multi-modal information

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

How to organize and retrieve images is now a great challenge in various domains. Image clustering is a key tool in some practical applications including image retrieval and understanding. Traditional image clustering algorithms consider a single set of features and use ad hoc distance functions, such as Euclidean distance, to measure the similarity between samples. However, multi-modal features can be extracted from images. The dimension of multi-modal data is very high. In addition, we usually have several, but not many labeled images, which lead to semi-supervised learning. In this paper, we propose a framework of image clustering based on semi-supervised distance learning and multi-modal information. First we fuse multiple features and utilize a small amount of labeled images for semi-supervised metric learning. Then we compute similarity with the Gaussian similarity function and the learned metric. Finally, we construct a semi-supervised Laplace matrix for spectral clustering and propose an effective clustering method. Extensive experiments on some image data sets show the competent performance of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Xia, H., Wu, P., Hoi, S.C.H.: Online multi-modal distance learning for scalable multimedia retrieval. In: Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pp. 455–464 (2013)

  2. Lu, Z.D., Leen, T.K.: Semi-supervised clustering with pairwise constraints: a discriminative approach. J. Mach. Learn. Res. 2, 299–306 (2007)

    Google Scholar 

  3. Basu, S., Bilenko, M., Mooney, R.J.: A probabilistic framework for semi-supervised clustering. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, vol. 8, pp. 59–68 (2004)

  4. Kulis, B., Bas, S., Dhillo, I., Moone, R.: Semi-supervised graph clustering: a kernel approach. In: Proceedings of the 22nd International Conference on Machine Learning, vol. 8, pp. 457–464 (2005)

  5. El Demerdash, O., Kosseim, L., Bergler, S.: Text-Based Clustering of the ImageCLEFphoto Collection for Augmenting the Retrieved Results. In: Advances in Multilingual and Multimodal Information Retrieval, pp. 562–568 (2008)

  6. Rahmani, R., Goldman, S.A., Zhang, H., Cholleti, S.R., Fritts, J.E.: Localized content-based image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 11, 1902–1912 (2008)

    Article  Google Scholar 

  7. Smeulders, A.W.M., Worring, M., Santini, S., Gupta, A., Jain, R.: Content-based image retrieval at the end of the early years. IEEE Trans. Pattern Anal. Mach. Intell. 12, 1349–1380 (2000)

    Article  Google Scholar 

  8. Manjunath, B.S., Ma, W.-Y.: Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 8, 837–842 (1996)

    Article  Google Scholar 

  9. Yang, J., Jiang, Y.-G., Hauptmann, A.G., Ngo, C.-W.: Evaluating bag-of- visual-words representations in scene classification. In: Proceedings of the International Workshop on Multimedia Information Retrieval, pp. 197–206 (2007)

  10. Wu, L., Hoi, S.C., Yu, N.: Semantics-preserving bag-of-words models and applications. IEEE Trans. Image Process. 7, 1908–1920 (2010)

    MathSciNet  Google Scholar 

  11. Maheshwari, M., Silakari, S., Motwani, M.: Image clustering using color and texture. In: First International Conference on Computational Intelligence, Communication Systems and Networks, Indore, India, pp. 403–408 (2009)

  12. Hammouche, K., Diaf, M., Postaire, J.-G.: A clustering method based on multidimensional texture analysis. Pattern Recognit. 1, 1265–1277 (2006)

    Article  Google Scholar 

  13. Antonopoulos, P., Nikolaidis, N., Pitas, I.: Hierarchical face clustering using sift image features. In: Proceedings of the 2007 IEEE Symposium on Computational Intelligence in Image and Signal Processing, Honolulu, HI, pp. 325–329 (2007)

  14. Fang, Y., Tan, T., Wang, Y.: Fusion of global and local features for face verification. In: 16th International Conference on Pattern Recognition, pp. 382–385 (2002)

  15. Wang, X., Tang, X.: Using random subspace to combine multiple features for face recognition. In: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 284–289 (2004)

  16. Fu, Y., Cao, L., Guo, G., Huang, T.S.: Multiple feature fusion by subspace learning. In: ACM International Conference on Image and Video Retrieval, pp. 127–134 (2008)

  17. Crammer, K., Kearns, M., Wortman, J.: Learning from multiple sources. Journal of Machine Learning Research 9, 1757–1774 (2008)

    MathSciNet  MATH  Google Scholar 

  18. Basu, S.: Semi-supervised clustering with limited background knowledge. In: Proceedings of the Ninth AAAI/SIGART Doctoral Consortium, vol. 7, pp. 979–980 (2004)

  19. Meng, L., Hwee Tan, A., Xu, D.: Semi-supervised heterogeneous fusion for multimedia data co-clustering. IEEE Trans. Knowl. Data Eng. 3, 1–14 (2013)

    Google Scholar 

  20. Kumar, N., Kummamuru, K., Paranjpe, D.: SemiCsupervised clustering with metric learning using relative comparisons. IEEE Trans. Knowl. Data Eng. 4, 496–503 (2008)

    Article  Google Scholar 

  21. Tang, W., Xiong, H., Zhong, S., Wu, J.: Enhancing semi-supervised clustering: a feature projection perspective. In: Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 707–716 (2007)

  22. Wagstaff, K., Cardie, C., Rogers, S., Schroedl, S.: Constrained Kmeans clustering with background knowledge. In: Proceedings of the 18th International Conference on Machine Learning, San Fransisco, pp. 577–584 (2001)

  23. Basu, S., Banerjee, A., Mooney, R.: Semi-supervised clustering by seeding. In: Proceedings of the 19th International Conference on Machine Learning, Sydney, Australia, pp. 19–26 (2002)

  24. McFee, B., Lanckriet, G.: Learning multi-modal similarity. J. Mach. Learn. Res. 12, 491–523 (2011)

    MathSciNet  MATH  Google Scholar 

  25. Niyogi, P.: Manifold regularization and semi-supervised learning: some theoretical analyses. J. Mach. Learn. Res. 14, 1229–1250 (2013)

    MathSciNet  MATH  Google Scholar 

  26. Jun Zha, Z., Mei, T., Wang, M., Wang, Z., Sheng Hua, X.: Robust distance metric learning with auxiliary knowledge. In: Proceedings of the 21st International Jont Conference on Artifical Intelligence, pp. 1327–1332 (2009)

  27. Ying, Y., Li, P.: Distance metric learning with eigenvalue optimization. J. Mach. Learn. Res. 13, 1–26 (2012)

    MathSciNet  MATH  Google Scholar 

  28. Fouad, S., Tino, P., Raychaudhury, S., Schneider, P.: Incorporating privileged information through metric learning. IEEE Trans. Neural Netw. Learn. Syst. 7, 1086–1098 (2013)

    Article  Google Scholar 

  29. Lim, D.K.H., McFee, B., Lanckriet, G.: Robust structural metric learning. In: Proceedings of the 30th International Conference on Machine Learning, pp. 615–623 (2013)

  30. Hoi, S.C.H., Liu, W., Fu Chang, S.: Semi-supervised distance metric learning for collaborative image retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2008)

  31. Bellet, A., Habrard, A., Sebban, M.: A survey on metric learning for feature vectors and structured data. Technical report (2013)

  32. Zha, Z.-J., Mei, T., Wang, M., Wang, Z.F., Hua, X.-S.: Robust distance metric learning with auxiliary knowledge. In: International Joint Conference on Artificial Intelligence, pp. 1327–1332 (2009)

  33. Niu, G., Dai, B., Yamada, M., Sugiyama, M.: Information-theoretic semi-supervised metric learning via entropy regularization. In: Proceedings of the 29th International Conference on Machine Learning, pp. 89–96 (2012)

  34. Baghshah, M.S., Shouraki, S.B.: Semi-supervised metric learning using pairwise constraints. In: Proceedings of the 21st International Jont Conference on Artifical Intelligence, pp. 1217–1222 (2009)

  35. Bucak, S.S., Jin, R., Jain, A.K.: Multiple kernel learning for visual object recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 11, 1–20 (2013)

    Google Scholar 

  36. Dalal, N., Triggs, B.: Histograms of Oriented Gradients for Human Detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2005)

  37. Sturm, J.F.: Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optim. Methods Softw. 11–12, 625–653 (1999)

    Article  MathSciNet  Google Scholar 

  38. Xia, H., Hoi, S.C.H., Jin, R., Zhao, P.L.: Online multiple kernel similarity learning for visual search. IEEE Trans. Pattern Anal. Mach. Intell. 1, 1–14 (2012)

    Google Scholar 

  39. Wu, P.C., Hoi, S.C.H., Xia, H., Zhao, P.L., Wang, D.Y., Miao, C.Y.: Online multimodal deep similarity learning with application to image retrieval. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 153–162 (2013)

  40. von Luxburg, U.: A tutorial on spectral clustering. Stat. Comput. 17(4), 395–416 (2007)

    Article  MathSciNet  Google Scholar 

  41. Chen, W.Y., Song, Y.Q., Bai, H.J., Lin, C.J., Chang, E.Y.: Parallel spectral clustering in distributed systems. IEEE Trans. Pattern Anal. Mach. Intell. 3, 568–586 (2011)

    Article  Google Scholar 

  42. Columbia University Image Library (COIL 20). http://www.cs.columbia.edu/CAVE/software/softlib/coil-20.php (1996)

  43. Oxfold Flower. http://www.robots.ox.ac.uk/~vgg/data0.html (2006)

  44. Li, F.F., Fergus, R., Member, S., Perona, P.: One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell. 4, 594–611 (2006)

    Google Scholar 

  45. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 12, 2323–2326 (2000)

    Article  Google Scholar 

  46. Hoi, S.C.H., Liu, W., Lyu, M.R., Ma, W.-Ying: Learning distance metrics with contextual constraints for image retrieval. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2072–2078 (2006)

  47. Sugiyama, M., Yamada, M., Kimura, M., Hachiya, H.: On information-maximization clustering: tuning parameter selection and analytic solution. In: Proceedings of the 28th International Conference on Machine Learning, Bellevue, USA (2011)

  48. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 8, 888–905 (2000)

    Google Scholar 

  49. McFee, B., Lanckriet, G.: Partial order embedding with multiple kernels. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 721–728. ACM, New York (2009)

  50. McFee, B., Lanckriet, G.: Learning multi-modal similarity. J. Mach. Learn. Res. 12, 491–523 (2011)

    MathSciNet  MATH  Google Scholar 

  51. Lu, J., Wang, G., Moulin, P.: Image classification using holistic multiple order statistics features and localized multi-kernel metric learning. In: IEEE International Conference on Computer Vision, pp. 329–336 (2013)

Download references

Acknowledgments

This work was partly supported by National Program on Key Basic Research Project (under Grant 2013CB329304), National Natural Science Foundation of China (under Grants 61222210 and 61432011), and New Century Excellent Talents in University (under Grant NCET-12-0399).

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory of Cognitive Computing and Application, School of Computer Science and Technology, Tianjin University, Tianjin, 300072, China

    Jianqing Liang, Yahong Han & Qinghua Hu

Authors
  1. Jianqing Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yahong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qinghua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qinghua Hu.

Additional information

Communicated by M. Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liang, J., Han, Y. & Hu, Q. Semi-supervised image clustering with multi-modal information. Multimedia Systems 22, 149–160 (2016). https://doi.org/10.1007/s00530-014-0433-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-014-0433-6

Keywords

Search

Navigation