Skip to main content
SpringerLink
Log in

Unified Multimodal Search Framework for Multimedia Information Retrieval

  • Conference paper
  • First Online:
Advanced Techniques in Computing Sciences and Software Engineering

Abstract

There is a trend towards construction of multimedia digital information resources which may hold diverse data types in the form of image, graphics, audio, video, and text based retrieval artifacts or objects. WWW is a huge multimedia information resource. Existing search mechanisms available on WWW are mostly mono-modal. Multimedia information needs are partially satisfied by using mono-modal search mechanisms. Multiple modalities of information are associated with multimedia retrieval object so multimodal search mechanisms are required for searching multimedia information resources. Existing search mechanisms available on WWW whether they are general purpose, domain specific or media specific partially fulfil multimedia information needs because they are mostly mono-modal and able to perform search with in one media type at a time. We explore with the help of multimedia object analysis that search with in all the subsets of multimedia object types is possible by using only four index types, two models of interactions and fifteen possible modes at most. A framework based on an architectural approach is proposed for multimedia information retrieval. Proposed retrieval framework is implemented. Present implementation gives query formulation and information presentation for all modes and search functionality with in few modes of multimedia search framework.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. B. Marshall, Y. Zhang, R. Shen, E. Fox & L.N. Cassel, “Convergence of Knowledge Management and E-Learning: the GetSmart Experience”, In IEEE’03, 3 rd ACM/IEEE-CS Joint Conference on Digital Libraries Digital Libraries, pp. 135–146, 2003.

    Google Scholar 

  2. L. Taycher, M. L. Cascia, & S. Sclaroff ” Image Digestion and Relevance Feedback in the ImageRover WWW Search Engine”, In VISUAL’06, pp.1–7, 1997.

    Google Scholar 

  3. N. Chen, “A Survey of Indexing and Retrieval of Multimodal Documents: Text and Images”, (2006–505). Canada, Queen’s University. 2006.

    Google Scholar 

  4. J.A. Arias & J.A. Sánchez, Content-Based Search and Annotations in Multimedia Digital Libraries. In ENC’03, 4 th Mexican International Conference on Computer Science, pp. 109–116, 2003.

    Google Scholar 

  5. M.G. Christel. Evaluation and User Studies with Respect to Video Summarization and Browsing. In Multimedia Content Analysis, Management, and Retrieval”, part of the IS&T/SPIE Symposium on Electronic Imaging, Vol. 607, 2006.

    Google Scholar 

  6. M.C. Schraefel, M. Wilson, A. Russel, & D.A. Smith. MSPACE: Improving Information Access to Multimedia Domains with Multimodal Exploratory Search. Communication of the ACM, 49(4), pp. 47–49, 2006.

    Article  Google Scholar 

  7. J. Yang, Y. Zhuang, & Q. Li. Multi-Modal Retrieval for Multimedia Digital Libraries: Issues, Architecture, and Mechanisms. In ICME’01, Workshop in Multimedia Communications and Computing, 2001.

    Google Scholar 

  8. U. Rashid, & M.A Bhatti. “Exploration and Management of Web Based Multimedia Information Resources”. In SCS2 07, 3rd International Conference on Systems, Computing Sciences and Software Engineering, pp. 500–507, 2007.

    Google Scholar 

  9. G. Marchionini & G. Geisler. The Open Video Digital Library. D-Lib Magazine, Volume 8 Number 12, 2002.

    Google Scholar 

  10. Wilfred Owen digital archive to expand. BLUEPRINT the newsletter of the University of Oxford, Oxford Press, pp. 4, 1993.

    Google Scholar 

  11. W.Dunn . EVIADA: Ethnomusicological Video for Instruction and Analysis Digital Archive. In JCDL’05, Joint Conference on Digital Libraries, pp. 40, 2005.

    Google Scholar 

  12. A. Kerne, E Koh, B. Dworaczyk, J.M. Mistrot, H. Choi, S.M. Smith, R. Graeber, D. Caruso, A. Webb, R. Hill & J. Albea. CombinFormation: A Mixed-Initiative System for Representing Collections as Compositions of Image and Text Surrogates. In JCDL’06, 6 th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 11–20, 2006.

    Google Scholar 

  13. I.H. Witten & D. Bainbridge. Building Digital Library Collections with Greenstone, In JCDL’05, 5th ACM/IEEE-CS Joint Conference on Digital Libraries, pp. 425–425, 2005.

    Google Scholar 

  14. D.Bainbridge, J.Thompson & I.H. Witten. Assembling and Enriching Digital Library Collections. In IEEE ’03, Joint Conference on Digital Libraries, pp. 323–334, 2003.

    Google Scholar 

  15. Wilson,. Advanced Search Interfaces considering Information Retrieval and Human Computer Interaction. Agents and Multimedia Group School of Electronics and Computer Science Faculty of Engineering and Applied Science University of Southampton. Retrieved from web site: www.iam.ecs.soton.ac.uk, 2007.

  16. E. Bruno & N.M. Loccoz. Interactive video retrieval based on multimodal dissimilarity representation. In MLMM’05, workshop on Machine Learning Techniques for Processing Multimedia Conten, 2005.

    Google Scholar 

  17. T. Westerveld, A.P.D. Vries, A.R.V. Ballegooij, F.M.G.D. Jong & D. Hiemstra. A Probabilistic Multimedia Retrieval Model and its Evaluation. In EURASIP’03, Journal on Applied Signal Processing, special Issue on Unstructured Information Management from Multimedia Data Sources, 2003(3), pp. 186–19, 2003.

    Google Scholar 

  18. W Agheal, & A.S Kellerman. Distributed Multimedia Technologies, Applications, and Opportunities in the Digital Information Industry, New York, ADDISON-WESLEY PUPLISHING COMPANY, 1996.

    Google Scholar 

  19. J.C. French, A.C. Chapin & W.N. Martin. An Application of Multiple Viewpoints to Content-Based Image Retrieval. In IEEE ’03, Joint Conference on Digital Libraries, pp. 128–130, 2003.

    Google Scholar 

  20. S.Y. Neo, J. Zhao, M.Y. Kan, & T.S. Chua. Video Retrieval using High Level Features: Exploiting Query Matching and Confidence-based Weighting. In CIVR’06, Proceedings of the Conference on Image and Video Retrieval, pp. 143–15, 2006.

    Google Scholar 

  21. U. Rashid, & M.A. Bhatti. 2007. Enhanced Term-specific Smoothing using Document World Clusters. In CITA’07,5 th International Conference on Information Technology in Asia, pp. 140–144, 2007.

    Google Scholar 

  22. D.Hiemstra. Term-Specific Smoothing for the Language Modelling Approach to Information Retrieval: The Importance of a Query Term. In SIGIR’02, Special Interest Group on Information Retrieval, pp. 35–41, 2006.

    Google Scholar 

  23. I. Somerville. “Software Engineering”, 7th edition, Pearson Addison Wesley.2004.

    Google Scholar 

  24. L. Bass, P. Clements, , & R. Kazman. “Software Architecture in Practice”, 2nd edition, Addison Wesley, 2002.

    Google Scholar 

  25. B. Fields, P. Wright, & Harrison. “Resource Modeling”. In proceedings of APCHI’96, pp. 181–191, 1996.

    Google Scholar 

  26. I. Heywood, S. Cornelius, , & S. Cerver. “An Introduction to Geographical Information Systems”, Addision Wesley, 1998.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computing, Riphah International University, Islamabad, Pakistan

    Umer Rashid & Iftikhar Azim Niaz

  2. Department of Computer Science, Quaid-i-Azam University, Islamabad, Pakistan

    Muhammad Afzal Bhatti

Authors
  1. Umer Rashid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Iftikhar Azim Niaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Afzal Bhatti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Umer Rashid .

Editor information

Editors and Affiliations

  1. School of Engineering, University of Bridgeport, University Avenue 221, Bridgeport, 06604, U.S.A.

    Khaled Elleithy

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media B.V.

About this paper

Cite this paper

Rashid, U., Niaz, I.A., Bhatti, M.A. (2010). Unified Multimodal Search Framework for Multimedia Information Retrieval. In: Elleithy, K. (eds) Advanced Techniques in Computing Sciences and Software Engineering. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-3660-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-90-481-3660-5_22

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-3659-9

  • Online ISBN: 978-90-481-3660-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation