Skip to main content
Springer Nature Link
Log in

Similarity-Based Retrieval of Temporal Specifications and its Application to the Retrieval of Multimedia Documents

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

In this paper, we describe a similarity-based retrieval framework for temporal information, such as multimedia presentations. We develop techniques that allow users to query and retrieve multimedia documents, based on their temporal content. For this purpose, we describe different temporal data models and a set of similarity metrics applicable for different retrieval tasks. We develop algorithms that efficiently compute these metrics and report on experiment results. We also develop algorithms that efficiently index temporal structures based on these measures and show that the proposed variant of multi dimensional scaling is efficient and provides high quality retrieval of temporal specifications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Adali, K.S. Candan, Su-Shing Chen, K. Erol, and V.S. Subrahmanian, “Advanced video information system: Data structures and query processing,” ACM-Springer Multimedia Systems Journal, Vol. 4, pp. 172–186, Aug. 96.

    Google Scholar 

  2. S. Adali, M.-L. Sapino, and V.S. Subrahmanian, “A multimedia presentation algebra,” SIGMOD’99, pp. 121–132, June 1999.

  3. J.F. Allen, “Maintaining knowledge about temporal intervals,” Communications of the ACM, Vol. 26, No. 11, pp. 832–843, Nov. 1983.

    Article  Google Scholar 

  4. J.F. Allen, “Towards a general theory of time and action,” Artificial Intelligence, Vol. 23, pp. 123–154, 1984.

    Article  Google Scholar 

  5. B. Aspavall and Y. Shiloach, “A polynomial time algorithm for solving systems of linear inequalities with two variables per inequality,” SIAM Journal of Computing, Vol. 9, No. 4, pp. 827–845, Nov. 1980.

    Article  Google Scholar 

  6. C. Baral, G. Gonzalez, and T.C. Som, “Design and implementation of display specifications for multimedia answers,” ICDE’98, Orlando, Florida, 1998, pp. 558–565.

    Google Scholar 

  7. A. Del Bimbo, E. Vicario, and D. Zingoni, “Symbolic description and visual querying of image sequences using spatio-temporal logic,” TKDE, Vol. 7, No. 4, pp. 609–622, 1995.

    Article  Google Scholar 

  8. M.C. Buchanan and P.T. Zellweger, “Automatic temporal layout mechanisms,” ACM Multimedia 93, pp. 341–350, 1993.

  9. M.C. Buchanan and P.T. Zellweger, “Scheduling multimedia documents using temporal constraints,” in Network and Operating System Support for Digital Audio and Video, P. Venkat Rangan (Ed.), Springer-Verlag: Berlin; Heidelberg: New York, 1993, pp. 237–249.

    Google Scholar 

  10. M.C. Buchanan and P.T. Zellweger, “Automatically generating consistent schedules for multimedia documents,” ACM/Springer-Verlag Journal of Multimedia Systems, Vol. 1, No. 2, 1993.

  11. K.S. Candan, “A framework for distributed multimedia collaborations,” Ph.D. Thesis, University of Maryland at College Park.

  12. K.S. Candan and R. Cutler, “Multimedia authoring systems,” Appeared as a chapter in the book Multimedia Database Systems, V.S. Subrahmanian and S. Jajoida (Eds.), Springer-Verlag.

  13. K.S. Candan, E. Hwang, B. Prabhakaran, and V.S. Subrahmanian, Resource Lock Commit Protocol (RLCP) for Multimedia Object Retrieval, Technical Report, UMCP-CSD CS-TR-3746, UMIACS-TR-97-08, Feb. 1997.

  14. K.S. Candan, E. Hwang, and V.S. Subrahmanian, “An event-based model for continuous media data on heterogeneous disk servers,” Accepted for publication in ACM-Springer Multimedia Systems Journal.

  15. K.S. Candan, E. Lemar, and V.S. Subrahmanian, “Management and rendering of multimedia views,” Fourth International Workshop on Multimedia Information Systems, 1998.

  16. K.S. Candan, E. Lemar, and V.S. Subrahmanian, “View management in multimedia databases,” VLDB Journal Vol. 9, No. 2, pp. 131–153, 2000.

    Google Scholar 

  17. K.S. Candan and W.-S. Li, “On similarity measures for multimedia database applications,” Knowledge and Information Systems, Vol. 3, No. 1, pp. 30–51, 2001.

    Google Scholar 

  18. K.S. Candan, W.-S. Li, and M.L. Priya, “Similarity-based ranking and query processing in multimedia databases,” DKE, Vol. 35, No. 3, pp. 259–298, 2000.

    Article  Google Scholar 

  19. K.S. Candan, B. Prabhakaran, and V.S. Subrahmanian, “A framework for supporting multimedia document authoring and presentation,” in the Fourth ACM International Multimedia Conference ‘96, Boston, Nov. 1996, pp. 329–339.

  20. K.S. Candan, B. Prabhakaran, and V.S. Subrahmanian, “Collaborative multimedia documents: Authoring and presentation,” Special issue of the International Journal of Intelligent Systems on Multimedia Computing Systems, Vol. 13, No. 12, 1998.

  21. K.S. Candan, B. Prabhakaran, and V.S. Subrahmanian, “Retrieval schedules based on resource availability and flexible presentation specifications,” ACM-Springer Multimedia Systems Journal, Vol. 6, No. 4, pp. 232–250, 1998.

    Article  Google Scholar 

  22. K.S. Candan, B. Prabhakaran, and V.S. Subrahmanian, “Collaborative multimedia documents: Authoring and presentation,” Accepted for publication in the Special Issue of the International Journal of Intelligent Systems on Multimedia Computing Systems.

  23. K.S. Candan, V.S. Subrahmanian, and P.V. Rangan, “Towards a theory of collaborative multimedia,” IEEE International Conference on Multimedia Computing and Systems, Hiroshima, Japan, June 96, pp. 279–282.

  24. K.S. Candan, V.S. Subrahmanian, and P. Venkat Rangan, “Collaborative multimedia systems: Synthesis of media objects,” IEEE Trans. on Knowledge and Data Engineering, Vol. 10, No. 3, pp. 433–457, May–June 1998.

    Article  Google Scholar 

  25. S. Chawathe, “Comparing hierarchical data in external memory,” to appear in Proceedings of the Twenty-fifth International Conference on Very Large Data Bases, Edinburgh, Scotland, U.K., 1999.

  26. S. Chawathe and H. Garcia-Molina, “Meaningful change detection in structured data,” in Proceedings of the ACM SIGMOD International Conference on Management of Data, Tucson, Arizona, May 1997, pp. 26–37

  27. Claudio S. Pinhanez and Aaron F. Bobick, “Fast constraint propagation on specialized allen networks and its application to action recognition and control,” Submitted to AAAI’98. Perceptual Computing Tech. Rep. No. 456. January 1998.

  28. Claudio S. Pinhanez, Kenji Mase, and Aaron F. Bobick, “Interval scripts: A design paradigm for story-based interactive systems,” CHI 1997, pp. 287–294, 1997.

  29. R. Dechter, I. Meiri, and J. Pearl, “Temporal constraint networks,” Artificial Intelligence, Vol. 49, pp. 61–95, 1991.

    Article  MathSciNet  Google Scholar 

  30. M.L. Escobar-Molano, D.A. Barrett, Z. Genova, and L. Zhang, “Retrieval scheduling for multimedia presentations,” Workshop on Multimedia Information Systems, -Capri, Italy, Nov. pp. 7–9, 2001.

    Google Scholar 

  31. R. Fagin and L. Stockmeyer, “Relaxing the triangle inequality in pattern matching,” International Journal of Computer Vision, Vol. 28, No. 3, pp. 219–231, 1998.

    Google Scholar 

  32. C. Faloutosos and Kin-Ip (David) Lin, “Fast map: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets,” ACM SIGMOD, San Jose, CA, pp. 163–174, May 1995.

    Google Scholar 

  33. M. Farach and M. Thorup, “Sparse dynamic programming for evolutionary-tree comparison,” SIAM Journal on Computing, Vol. 26, No. 1, pp. 210–230, 1997.

    Article  Google Scholar 

  34. V. Hakkoymaz, J. Kraft, and G. Özsoyo$lu, “Constraint-based automation of multimedia presentation assembly,” ACM Multimedia Systems Journal, to appear, 1998.

  35. V. Hakkoymaz and G. Özsoyo$lu, “A constraint-driven approach to automate the organization and playout of presentations in multimedia databases,” Journal of Multimedia Tools and Applications, Kluwer Pub., Vol. 4, pp. 171–197, 1997.

    Article  Google Scholar 

  36. R. Hamakawa and J. Reikomoto, “Object composition and playback models for handling multimedia data” in ACM Multimedia Conference ‘93, 1993, pp. 273–281.

  37. http://www.w3.org/TR/REC-smil/

  38. http://www.macromedia.com/

  39. http://netlib.bell-labs.com/netlib/master/readme.html

  40. ISO 13522-5.

  41. ISO 8879:1986.

  42. ISO/IEC 10744.

  43. L. Li, A. Karmouch, and N.D. Georganas, “Multimedia teleorchestra with independent sources: Part 1 and Part 2,” ACM/Springer-Verlag Journal of Multimedia Systems, Vol. 1, No. 4, pp. 143–165, 1994.

    Article  Google Scholar 

  44. M.Y. Kim and J. Song, “Hyperstories: Combining time, space and asynchrony in multimedia documents,” RC19277(83726) (revised 1995) IBM Computer Science/Mathematics Research Report, 1993.

  45. M.Y. Kim and J. Song, “Multimedia documents with elastic time,” ACM Multimedia Conference ‘95, 1995.

  46. J.B. Kruskal, “Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis,” Psychometrika, Vol. 29, No. 1, pp. 1–27, March 1964.

    Google Scholar 

  47. J.B. Kruskal, “Nonmetric multidimensional scaling: A numerical method,” Psychometrika, Vol. 29, No. 2 pp. 115–129, 1964.

    Google Scholar 

  48. J.B. Kruskal and Myron Wish, “Multidimensional scaling,” SAGE publications, Beverly Hills, 1978.

    Google Scholar 

  49. W.-S. Li, K.S. Candan, K. Hirata, and Y. Hara, “Hierarchical image modeling for object-based media retrieval,” Data and Knowledge Engineering, Elsvier Science, Vol. 27, No. 2, pp. 139–176, 1998.

    Google Scholar 

  50. W.-S. Li, K.S. Candan, K. Hirata, and Y. Hara, “Supporting efficient multimedia database exploration,” VLDB Journal, Vol. 9, No. 4, pp. 312–326, 2001.

    Google Scholar 

  51. R.M. de Lima, P.A. da S. Gonçalves, F.P. Junqueria, and O.C.M.B. Duarte, “SAMM: An integrated environment to support multimedia synchronization of pre-orchestrated data,” ICMCS’99, pp. 929–933, June 1999.

  52. T.D.C. Little and A. Ghafoor, “Synchronization and storage models for multimedia objects,” IEEE J. on Selected Areas of Communications, Vol. 8, No. 3, pp. 413–427, 1990.

    Article  Google Scholar 

  53. T.D.C. Little and A. Ghafoor, “Interval-based conceptual models for time-dependent multimedia data” IEEE Transactions on Knowledge and Data Engineering, Vol. 5, No. 4, pp. 551–563, 1993.

    Article  Google Scholar 

  54. F. Luccio and L. Pagli, “Approximate matching for two families of trees,” Information and Computation, Vol. 123, No. 1, pp. 111–120, 1995.

    Article  Google Scholar 

  55. I. Mirbel, B. Pernici, and M. Vazirgiannis, “Temporal integrity constraints in interactive multimedia documents,” ICMCS’99, pp. 867–871, June 1999.

  56. E. Myers, “An O(ND) difference algorithms and its variations,” Algorithmica, Vol. 1, No. 2, pp. 251–266, 1986.

    Article  Google Scholar 

  57. S. Nestorov, J. Ullman, J. Wiener, and S. Chawathe, “Representative objects: Concise representations of semistructured, hierarchical data,” in Proceedings of the International Conference on Data Engineering, U.K., 1997, pp. 79–90.

  58. G. Özsoyo$lu, V. Hakkoymaz, and J.D. Kraft, “Automating the assembly of presentations from multimedia databases,” in Twelfth International Conference on Data Engineering, New Orleans, Feb. 1996, pp. 593–601.

  59. B. Prabhakaran and S.V. Raghavan, “Synchronization models for multimedia presentation with user participation,” ACM/Springer-Verlag Journal of Multimedia Systems, Vol. 2, No. 2, pp. 53–62, 1994. Also in the Proceedings of the First ACM Conference on Multimedia Systems, Anaheim, California, Aug. 1993, pp. 157–166.

    Article  Google Scholar 

  60. S. Selkow, “The tree-to-tree editing problem,” Information Processing Letters, Vol. 6, No. 6, pp. 184–186, 1977.

    Article  Google Scholar 

  61. D. Shasha and K. Zhang, “Fast algorithms for the unit cost distance between trees,” Journal of Algorithms, Vol. 11, pp. 581–621, 1990.

    Article  Google Scholar 

  62. J. Song, Y.N. Doganata, M.Y. Kim, and A.N. Tantawi, “Modeling timed user-interactions in multimedia documents,” IBM Computer Science Research Report, 1995. Also appeared in IEEE International Conference on Multimedia Computing and Systems ‘96, Hiroshima, June 1996, pp. 407–416.

  63. Y. Song, M. Mielke, and A. Zhang, “NetMedia: Synchronized streaming of multimedia presentations in distributed environments,” ICMCS’99, pp. 585–590, June 1999.

  64. D. Suciu, “Semistructured data and XML,” in Proceedings of 5th International Conference of Foundations of Data Organization (FODO’98), Kobe, Japan, Nov. 1998.

  65. K.-C. Tai, “The tree-to-tree correction problem,” Journal of ACM, Vol. 26, pp. 422–433, March 1979.

    Article  Google Scholar 

  66. The AthenaMuse 2 Functional Specification, CECI(MIT) Report, May 1992.

  67. W.S. Torgerson, “Multidimensional scaling: I. theory and method,” Psycometrika, Vol. 17, pp. 401–419, 1952.

    Google Scholar 

  68. P. van Beek, “Approximation algorithms for temporal reasoning,” in Proceedings of the 11th International Joint Conference on Artificial Intelligence. Morgan Kaufmann: Detroit, MI, USA, Aug. 1989, pp. 1291–1296.

  69. M. Vazirgiannis and S. Boll, “Events in interactive multimedia applications: Modelling and implementation design,” IEEE International Conference on Multimedia Computing and Systems, June 1997.

  70. M. Vilain and H. Kautz, “Constraint propagation algorithms for temporal reasoning,” in Proceedings of AAAI-86, Artificial Intelligence, Aug. 1986, pp. 377–382.

  71. S. Wirag, “Scheduling of adaptive multimedia documents,” ICMCS’99, pp. 307–311, June 1999.

  72. World Wide Web Consortium (W3C), “Synchronized multimedia integration language recommendation,” 15 June, 1998.

  73. World Wide Web Consortium (W3C), “Extensible Markup Language 1.0 Recommendation,” 10 Feb. 1998.

  74. www.oratrix.com/;GRiNS/gse/index.html, available as examples of GRiNS Version 1.0 for SMIL 1.0 player

  75. F. Yaman, S. Adali, D. Nau, M.L. Sapino, and V.S. Subrahmanian, Plan Databases: Model and Algebra submitted for publication.

  76. P. Yamuna and K.S. Candan, “Efficient similarity-based retrieval of temporal structures,” Symposium on Applications and the Internet, Jan. 2001.

  77. Prakash Yamuna and K.S. Candan, “Similarity based retrieval of temporal documents,” International Workshop on Multimedia Information Retrieval, Nov. 2000.

  78. K. Zhang and D. Shasha, “Simple fast algorithms for the editing distance between trees and related problems,” SIAM Journal of Computing, Vol. 18, No. 6, pp. 1245–1262, 1989.

    Article  Google Scholar 

  79. K. Zhang, J.T.L. Wang, and D. Shasha, “On the lediting distance between undirected acyclic graphs,” International Journal of Computer Science, Vol. 7, No.1, pp. 43–57, 1996.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science and Engineering Department, Arizona State University, Tempe, AZ, 85287, USA

    K. Selçuk Candan & Prakash Yamuna

Authors
  1. K. Selçuk Candan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Prakash Yamuna
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to K. Selçuk Candan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Candan, K.S., Yamuna, P. Similarity-Based Retrieval of Temporal Specifications and its Application to the Retrieval of Multimedia Documents. Multimed Tools Appl 27, 143–180 (2005). https://doi.org/10.1007/s11042-005-2717-5

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-005-2717-5

Keywords