Skip to main content
SpringerLink
Log in

A Novel Approach Towards Large Scale Cross-Media Retrieval

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

With the rapid development of Internet and multimedia technology, cross-media retrieval is concerned to retrieve all the related media objects with multi-modality by submitting a query media object. Unfortunately, the complexity and the heterogeneity of multi-modality have posed the following two major challenges for cross-media retrieval: 1) how to construct a unified and compact model for media objects with multi-modality, 2) how to improve the performance of retrieval for large scale cross-media database. In this paper, we propose a novel method which is dedicate to solving these issues to achieve effective and accurate cross-media retrieval. Firstly, a multi-modality semantic relationship graph (MSRG) is constructed using the semantic correlation amongst the media objects with multi-modality. Secondly, all the media objects in MSRG are mapped onto an isomorphic semantic space. Further, an efficient indexing MK-tree based on heterogeneous data distribution is proposed to manage the media objects within the semantic space and improve the performance of cross-media retrieval. Extensive experiments on real large scale cross-media datasets indicate that our proposal dramatically improves the accuracy and efficiency of cross-media retrieval, outperforming the existing methods significantly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Zhang H, Zhuang Y, Wu F. Cross-modal correlation learning for clustering on image-audio dataset. In Proc. the 15th ACM Int. Conf. Multimeida, September 2007, pp.273-276.

  2. Yang Y, Zhuang Y, Wu F, Pan Y. Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Transactions on Multimedia, 2008, 10(3): 437-446.

    Article  Google Scholar 

  3. Yang Y, Xu D, Nie F P et al. Ranking with local regression and global alignment for cross-meida retrieval. In Proc. the 17th ACM Int. Conf. Multimedia, Oct. 2009, pp.175-184.

  4. Lew M, Sebe N, Djeraba C, Jain R. Content-based multimedia information retrieval: State of the art and challenges. ACM TOMCCAP, 2006, 2(1): 1-19.

    Article  Google Scholar 

  5. Adams W, Iyengar G, Lin C Y et al. Semantic indexing of multimedia content using visual, audio and text cues. EURASIP J. Adv. Signal. Process, 2003, 10(2): 170-185.

    Article  Google Scholar 

  6. Kennedy L, Chang S F. A reranking approach for context-based concept fusion in video indexing and retrieval. In Proc. the 6th ACM CIVR, July 2007, pp.333-340.

  7. Hotelling H. Relations between two sets of variates. Biometrike, 1936, 28(3/4): 321-377.

    Article  MATH  Google Scholar 

  8. Chang S F, MaWY, Smeulders A. Recent advances and challenges of semantic image/video search. In Proc. ICASSP, April 2007, pp.12-16.

  9. Snoek C, Worring M. Multimodal video indexing: A review of the state-of-the-art. MTA, 2005, 25(1): 5-35.

    Google Scholar 

  10. Smeaton A F, Over P, Kraaij W. Evaluation campaigns and TRECVid. In Proc. the 8th ACM MIR, Oct. 2006, pp.321-330.

  11. Paramita M, Sanderson M, Clough P. Diversity in photo retrieval: Overview of the ImageCLEF photo task 2009. In Lecture Notes in Computer Science 6242, Peters C, Caputo B, Gonzalo J et al. (eds.), Springer-Verlag, 2009, pp.45-59.

  12. Naphade M, Smith J R, Tesic J, Chang S F, Hsu W, Kennedy L, Hauptmann A, Curtis J. Large-scale concept ontology for multimedia. IEEE Multimedia, 2006, 13(3): 86-91.

    Article  Google Scholar 

  13. Chen T, Cheng M M, Tan P et al. Sketch2Photo: Internet image montage. ACM TOG, 2009, 28(5), Article No. 124.

  14. Ajorloo H, Lakdashti A. HBIR: Hypercube-Based Image Retrieval. J. Comput. Sci. Technol., 2012, 27(1): 147-162

    Article  Google Scholar 

  15. Feng B L, Cao J, Bao X G et al. Graph-based multi-space semantic correlation propagation for video retrieval. The Visual Computer, 2011, 27(1): 21-34.

    Article  Google Scholar 

  16. Csurka G, Skaff S, Marchesotti L, Saunders C. Building look&feel concept models from color combinations with applications in image classification, retrieval, and color transfer. The Visual Computer, 2011, 27(12): 1039-1053.

    Article  Google Scholar 

  17. Guttman A. R-trees: A dynamic index structure for spatial searching. In Proc. SIGMOD, June 1984, pp.47-57.

  18. Weber G, Schek R, Blott H. A quantiative analysis and performance study for similarity search methods in high-dimensional spaces. In Proc. the 24th VLDB, August 1998, pp.194-205.

  19. Jagadish H V, Ooi B C, Tan K L et al. iDistance: An adaptive B+-tree based indexing method for nearest neighbor search. ACM TODS, 2005, 30(2): 364-397.

    Article  Google Scholar 

  20. Ciaccia P, Patella M, Zezula P. M-tree: An efficient access method for similarity search in metric spaces. In Proc. the 23rd VLDB, August 1997, pp.426-435.

  21. Ji H, Grishman R. Refining event extraction through cross-document inference. In Proc. the 46th ACL, Jun. 2008, pp.254-262.

  22. Ji H, Grishman R, Freitag D et al. Name extraction and translation for distillation. In Handbook of Natural Language Processing and Machine Translation, Olive J, Christianson C, John M (eds.), Springer, 2009, pp.21-29.

  23. Naphade M R, Kennedy L, Kender J et al. A light scale concept ontology for multimedia understanding for TRECVID 2005. Technical report RC23612, IBM, May 2005.

  24. Böhm C. A cost model for query processing in high dimensional data spaces. ACM TODS, 2000, 25(2): 129-178.

    Article  Google Scholar 

  25. Yanagawa A, Chang S F, Kennedy L, Hsu W. Columbia university's baseline detectors for 374 LSCOM semantic visual concepts. ADVENT Technical Report #222-2006-8, Columbia University, March 2007.

  26. Zhuang Y, Li Q, Chen L. A unified indexing structure for efficient cross-media retrieval. In Proc. the 14th DASFAA, April 2009, pp.677-692.

  27. Lu B, Wang G R, Yuan Y. Towards large scale cross-media retrieval via modeling heterogeneous information and exploring an efficient indexing scheme. In Proc. Computational Visual Media, Nov. 2012, pp.202-209.

Download references

Author information

Authors and Affiliations

  1. College of Information Science and Engineering, Northeastern University, Shenyang, 110819, China

    Bo Lu, Guo-Ren Wang (Member, CCF, ACM, IEEE) & Ye Yuan

Authors
  1. Bo Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guo-Ren Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ye Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Lu.

Additional information

This work was supported by the National Natural Science Foundation of China under Grant Nos. 61025007, 60933001, 61100024, the National Basic Research 973 Program of China under Grant No. 2011CB302200-G, the National High Technology Research and Development 863 Program of China under Grant No. 2012AA011004, and the Fundamental Research Funds for the Central Universities of China under Grant No. N110404011.

*The preliminary version of the paper was published in the Proceedings of the 2012 Computational Visual media Conference.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lu, B., Wang, GR. & Yuan, Y. A Novel Approach Towards Large Scale Cross-Media Retrieval. J. Comput. Sci. Technol. 27, 1140–1149 (2012). https://doi.org/10.1007/s11390-012-1292-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-012-1292-2

Keywords

Search

Navigation