Skip to main content

Advertisement

Springer Nature Link
Log in

A framework to explore results in multiple media information aggregated search

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

On web information exists in the form of text, audio, image, and video objects often referred to multiple media objects. Vertical web search provides the search of multiple media information usually via keyword-based queries. The search results in different media formats usually presented in separate panels/tabs; integration is mostly non-blended. Therefore, results exploration via vertical web search engines require the selection of a source and scrolling of a linear ranked list of results. Relationships in the results presented in separate panels/tabs are mostly not considered. Search aggregations unify results from several vertical web sources via blended integration, but exploration still requires scrolling of a linear ranked list. Multimedia search frameworks provide the exploration of results in different media formats but more focused towards the retrieval issues. We proposed a multiple media information search framework to address issues, particularly in aggregated search. Our search framework provides a mechanism to explore results via non-linear ways. The search framework realized by suggesting a framework architecture design and instantiating a search tool. The effectiveness of blended integration and browsing is measured via precision and click through rate respectively. Search task support in results exploration mechanism measured via task-based evaluation. We also validated the conformance of various search/exploration attributes discussed in the state-of-the-art in our frameworks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. http://www.dogpile.com/.

  2. https://duckduckgo.com/.

  3. https://www.ixquick.com/.

  4. https://www.google.com.pk/.

  5. http://www.isearch-project.eu/isearch/.

  6. http://vcl.iti.gr/multimodal-search-and-retrieval/.

  7. freely downloaded from http://chatzichristofis.info.

References

  1. Arguello J (2015) Improving aggregated search coherence. In: Proceedings of the 37th European conference on information retrieval. Springer, pp 25–36

  2. Axenopoulos A, Daras P, Malassiotis S, Croce V, Lazzaro M, Etzold J, Grimm P, Massari A, Camurri A, Steiner T et al (2012) I-search: a unified framework for multimodal search and retrieval. In: The future internet. Springer, pp 130–141

  3. Axenopoulos A, Manolopoulou S, Daras P (2011) Multimodal search and retrieval using manifold learning and query formulation. In: Proceedings of the 16th international conference on 3d web technology. ACM, pp 51–56

  4. Bozzon A, Fraternali P (2010) Multimedia and multimodal information retrieval. In: Search computing. Springer, pp 135–155

  5. Brooke J et al (1996) Sus-a quick and dirty usability scale. Usab Eval Ind 189(194):4–7

    Google Scholar 

  6. Cechticky V, Chevalley P, Pasetti A, Schaufelberger W (2003) A generative approach to framework instantiation. Lect Notes Comput Sci 267–286

  7. Chatzichristofis SA, Boutalis YS (2008) Cedd: color and edge directivity descriptor: a compact descriptor for image indexing and retrieval. In: Proceedings of the international conference on computer vision systems. Springer, pp 312–322

  8. Chatzichristofis SA, Zagoris K, Boutalis YS, Papamarkos N (2010) Accurate image retrieval based on compact composite descriptors and relevance feedback information. Int J Pattern Recognit Artif Intell 24(2):207–244

    Article  Google Scholar 

  9. Chechik G, Ie E, Rehn M, Bengio S, Lyon D (2008) Large-scale content-based audio retrieval from text queries. In: Proceedings of the 1st ACM international conference on multimedia information retrieval. ACM, pp 105–112

  10. Chen X, Hero III AO, Savarese S (2012) Multimodal video indexing and retrieval using directed information. IEEE Trans Multimed 14(1):3–16

    Article  Google Scholar 

  11. Chen Y, Sampathkumar H, Luo B, Chen XW (2013) ilike: bridging the semantic gap in vertical image search by integrating text and visual features. IEEE Trans Knowl Data Eng 25(10):2257–2270

    Article  Google Scholar 

  12. Chin JP, Diehl VA, Norman KL (1988) Development of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 213–218

  13. Crnkovic I, Sentilles S, Vulgarakis A, Chaudron MR (2011) A classification framework for software component models. IEEE Trans Softw Eng 37(5):593–615

    Article  Google Scholar 

  14. Daras P, Axenopoulos A, Darlagiannis V, Tzovaras D, Bourdon XL, Joyeux L, Verroust-Blondet A, Croce V, Steiner T, Massari A et al (2011) Introducing a unified framework for content object description. Int J Multimed Intell Secur 2(3):351–375

    Article  Google Scholar 

  15. Dias R, Fonseca MJ, Silva N, Cardoso T (2015) Encontra: a generic multimedia information retrieval meta-framework. Multimed Tools Appl 74(11):3691–3713

    Article  Google Scholar 

  16. Golovchinsky G, Diriye A, Dunnigan T (2012) The future is in the past: designing for exploratory search. In: Proceedings of the 4th symposium on information interaction in context. ACM, pp 52–61

  17. Hakkoymaz V (2010) Multimedia presentation organization and playout management using intelligent agents. Multimed Tools Appl 47(3):477–505

    Article  Google Scholar 

  18. Hanjalic A (2012) New grand challenge for multimedia information retrieval: bridging the utility gap. Int J Multimed Inf Retriev 1(3):139–152

    Article  Google Scholar 

  19. Hu W, Xie N, Li L, Zeng X, Maybank S (2011) A survey on visual content-based video indexing and retrieval. IEEE Trans Syst Man Cybern 41(6):797–819

    Article  Google Scholar 

  20. ISO M (2011) Systems and software engineering–architecture description. Tech. rep. ISO/IEC/IEEE 42010

  21. James N, Todorov K, Hudelot C (2011) Combining visual and textual modalities for multimedia ontology matching. Sem Multimed 95–110

  22. Jiang W, Loui AC (2012) Video concept detection by audio-visual grouplets. Int J Multimed Inf Retriev 1(4):223–238

    Article  Google Scholar 

  23. Kalamaras I, Mademlis A, Malassiotis S, Tzovaras D (2013) A novel framework for retrieval and interactive visualization of multimodal data. Electron Lett Comput Vis Image Anal 12(2):28–39

    Google Scholar 

  24. Kaszkiel M, Zobel J (2001) Effective ranking with arbitrary passages. J Amer Soc Inf Sci Technol 52(4):344–364

    Article  Google Scholar 

  25. Kiktova-Vozarikova E, Juhar J, Cizmar A (2015) Feature selection for acoustic events detection. Multimed Tools Appl 74(12):4213–4233

    Article  Google Scholar 

  26. Kim J (2009) Describing and predicting information-seeking behavior on the web. J Amer Soc Inf Sci Technol 60(4):679–693

    Article  Google Scholar 

  27. Kopliku A (2009) Aggregated search: from information nuggets to aggregated documents. In: Proceedings of the 6th french conference on information retrieval and applications, pp 495–502

  28. Kopliku A, Pinel-Sauvagnat K, Boughanem M (2014) Aggregated search: a new information retrieval paradigm. ACM Comput Surv 46(3):1–31

    Article  Google Scholar 

  29. Lauer C (2009) Contending with terms:multimodal and multimedia in the academic and public spheres. Comput Compos 26(4):225–239

    Article  Google Scholar 

  30. Lazaridis M, Axenopoulos A, Rafailidis D, Daras P (2013) Multimedia search and retrieval using multimodal annotation propagation and indexing techniques. Signal Process Image Commun 28(4):351–367

    Article  Google Scholar 

  31. Le TH, Elghazel H, Hacid MS (2012) A relational-based approach for aggregated search in graph databases. In: Proceedings of the international conference on database systems for advanced applications. Springer, pp 33–47

  32. Lewis JR (1995) Ibm computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Human-Comput Interact 7(1):57–78

    Article  Google Scholar 

  33. Li L, Li J (2011) Mqss: multimodal query suggestion and searching for video search. Multimed Tools Appl 54(1):55–68

    Article  Google Scholar 

  34. Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recog 40(1):262–282

    Article  MATH  Google Scholar 

  35. Luo H, Fan J, Zhou Y (2011) Multimedia news exploration and retrieval by integrating keywords, relations and visual features. Multimed Tools Appl 51(2):625–648

    Article  Google Scholar 

  36. Marchionini G (2006) Exploratory search: from finding to understanding. Commun ACM 49(4):41–46

    Article  Google Scholar 

  37. Mathias Filho I, de Oliveira TC, de Lucena CJ (2004) A framework instantiation approach based on the features model. J Syst Softw 73(2):333–349

    Article  Google Scholar 

  38. Messina A, Montagnuolo M (2009) A generalised cross-modal clustering method applied to multimedia news semantic indexing and retrieval. In: Proceedings of the 18th international conference on world wide web. ACM, pp 321–330

  39. Mitrović D, Zeppelzauer M, Breiteneder C (2010) Features for content-based audio retrieval. Adv Comput 78(3):71–150

    Article  Google Scholar 

  40. Natsev AP, Haubold A, Teṡić J, Xie L, Yan R (2007) Semantic concept-based query expansion and re-ranking for multimedia retrieval. In: Proceedings of the 15th international conference on multimedia. ACM, pp 991–1000

  41. Nielsen J, Landauer TK (1993) A mathematical model of the finding of usability problems. In: Proceedings of the INTERACT’93 and CHI’93 conference on human factors in computing systems. ACM, pp 206–213

  42. Normark CJ (2015) Design and evaluation of a touch-based personalizable in-vehicle user interface. Int J Human-Comput Interact 31(11):731–745

    Article  Google Scholar 

  43. Özacar T (2016) A tool for producing structured interoperable data from product features on the web. Inf Syst 56(3):36–54

    Article  Google Scholar 

  44. Perry DE, Wolf AL (1992) Foundations for the study of software architecture. In: ACM SIGSOFT Software engineering notes, vol 17. ACM, pp 40–52

  45. Pinto-Cáceres SM, Almeida J, Baranauskas MCC, Torres RDS (2015) Fisir: a flexible framework for interactive search in image retrieval systems. In: Lecture notes in computer science, vol. 8935. Springer, pp 335–347

  46. Rafailidis D, Manolopoulou S, Daras P (2013) A unified framework for multimodal retrieval. Pattern Recogn Lett 46(12):3358–3370

    Article  Google Scholar 

  47. Rashid U, Niaz IA, Bhatti MA (2009) M3l: architecture for multimedia information retrieval. In: Proceedings of the 6th international conference on information technology: new generations. IEEE, pp 1067–1072

  48. Rashid U, Niaz IA, Bhatti MA (2010) Unified multimodal search framework for multimedia information retrieval. In: Advanced techniques in computing sciences and software engineering. Springer, pp 129–136

  49. Rashid U, Viviani M, Pasi G (2016) A graph-based approach for visualizing and exploring a multimedia search result space. Inf Sci 370:303–322

    Article  Google Scholar 

  50. Rasiwasia N, Costa Pereira J, Coviello E, Doyle G, Lanckriet GR, Levy R, Vasconcelos N (2010) A new approach to cross-modal multimedia retrieval. In: Proceedings of the 18th international conference on multimedia. ACM, pp 251–260

  51. Richards M (2015) Software architecture patterns. OReilly, Sebastopol

  52. Rigamonti M, Lalanne D, Ingold R (2007) Faericworld: browsing multimedia events through static documents and links. In: Proceedings of the IFIP conference on human-computer interaction. Springer, pp 102–115

  53. Rizzo G, Steiner T, Troncy R, Verborgh R, Redondo García JL, Van de Walle R (2012) What fresh media are you looking for?: retrieving media items from multiple social networks. In: Proceedings of the international workshop on socially aware multimedia. ACM, pp 15–20

  54. Seifert C, Jurgovsky J, Granitzer M (2014) Facetscape: a visualization for exploring the search space. In: Proceedings of the 18th international conference on information visualization. IEEE, pp 94–101

  55. Shokouhi M, Si L (2011) Federated search. Found Trends Inf Retriev 5(1):1–102

    Article  Google Scholar 

  56. Slaney M (2011) Precision-recall is wrong for multimedia. IEEE MultiMed 18(3):4–7

    Article  Google Scholar 

  57. Smith DA, Owens A, Russell A, Harris C, Wilson M et al (2005) The evolving mspace platform: leveraging the semantic web on the trail of the memex. In: Proceedings of 6th ACM conference on hypertext and hypermedia. ACM, pp 174–183

  58. Snoek CG, Worring M (2005) Multimodal video indexing: a review of the state-of-the-art. Multimed Tools Appl 25(1):5–35

    Article  Google Scholar 

  59. Sommerville I (2004) Software engineering. International computer science series. Addison Wesley

  60. Steiner T, Sutton L, Spiller S, Lazzaro M, Nucci F, Croce V, Massari A, Camurri A, Verroust-Blondet A, Joyeux L et al (2012) I-search: a multimodal search engine based on rich unified content description (rucod). In: Proceedings of the 21st international conference companion on world wide web. ACM, pp 291–294

  61. Sushmita S (2012) Study of result presentation and interaction for aggregated search. University of Glasgow

  62. Sushmita S, Joho H, Lalmas M (2009) A task-based evaluation of an aggregated search interface. In: Proceedings of the international symposium on string processing and information retrieval. Springer, pp 322–333

  63. Sushmita S, Joho H, Lalmas M, Jose JM (2009) Understanding domain relevance in web search. In: Proceedings of the workshop on web search result summarization and presentation. WSSP

  64. Sushmita S, Joho H, Lalmas M, Villa R (2010) Factors affecting click-through behavior in aggregated search interfaces. In: Proceedings of the 19th ACM international conference on information and knowledge management. ACM, pp 519–528

  65. Tullis TS, Stetson JN (2004) A comparison of questionnaires for assessing website usability. In: Proceedings of the usability professional’s association conference. Usability professional association, pp 1–12

  66. Viaud ML, Thièvre J, Goëau H, Saulnier A, Buisson O (2008) Interactive components for visual exploration of multimedia archives. In: Proceedings of the 2008 international conference on content-based image and video retrieval. ACM, pp 609–616

  67. Vrochidis S, Papadopoulos S, Moumtzidou A, Sidiropoulos P, Pianta E, Kompatsiaris I (2010) Towards content-based patent image retrieval: a framework perspective. World Patent Inf 32(2):94–106

    Article  Google Scholar 

  68. Wang Z, Hoffman MD, Cook PR, Li K (2006) Vferret: content-based similarity search tool for continuous archived video. In: Proceedings of the 3rd ACM workshop on continuous archival and retrieval of personal experiences, pp 19–26. ACM

  69. Wengang C, De X (2002) Content-based video retrieval using audio and visual clues. In: Proceedings of IEEE conference on computers, communications, control and power engineering. IEEE, pp 586–589

  70. White RW, Roth RA (2009) Exploratory search: beyond the query-response paradigm. In: Synthesis lectures on information concepts, retrieval, and services, vol 1. Morgan & Claypool Publishers, pp 1–98

  71. Wilson TD (1999) Models in information behaviour research. J Document 55(3):249–270

    Article  Google Scholar 

  72. Yang J, Li Q, Zhuang Y (2002) Octopus: aggressive search of multi-modality data using multifaceted knowledge base. In: Proceedings of the 11th international conference on world wide web. ACM, pp 54–64

  73. Yang J, Zhuang Y, Li Q (2001) Search for multi-modality data in digital libraries. In: Proceedings of the pacific-rim conference on multimedia. Springer, pp 482–489

  74. Yang Y, Wu F, Xu D, Zhuang Y, Chia LT (2010) Cross-media retrieval using query dependent search methods. Pattern Recogn Lett 43(8):2927–2936

    Article  MATH  Google Scholar 

  75. Zhuang YT, Yang Y, Wu F (2008) Mining semantic correlation of heterogeneous multimedia data for cross-media retrieval. IEEE Trans Multimed 10(2):221–229

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to acknowledge the provision of research facility provided by Quaid-i-Azam University, Islamabad to carry out this research.

Author information

Authors and Affiliations

  1. Department of Computer Sciences, Quaid-i-Azam University, Islamabad, 45320, Pakistan

    Umer Rashid & M. Afzal Bhatti

Authors
  1. Umer Rashid
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. M. Afzal Bhatti
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Umer Rashid.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rashid, U., Bhatti, M.A. A framework to explore results in multiple media information aggregated search. Multimed Tools Appl 76, 25787–25826 (2017). https://doi.org/10.1007/s11042-017-4769-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4769-8

Keywords