Skip to main content
SpringerLink
Log in

Enhancing multimedia document modeling through extended orbit-based rhetorical structure: an approach to media weighting for importance determination

  • Regular paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

This paper proposes a graph-based approach to determine the importance of each media in a multimedia document by expanding the traditional four-dimensional model with a dimension that captures the rhetorical relations between different media types. The proposed approach utilizes an algorithm to weight the media types based on their significance. The use of rhetorical structure theory enables the determination of the significance of each media type, making it useful for document adaptation, automatic composition, and automatic generation of summaries. The approach utilizes an extended orbits-based rhetorical structure that is a novel method for determining the importance of media types in multimedia documents. The proposed approach is effective in capturing the importance of each media type and can be utilized in a wide range of applications, making it a potential solution to the limitations of the traditional model. This research has implications for a range of applications, including document adaptation, automatic composition, and automatic generation of summaries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 5

Similar content being viewed by others

References

  1. Ono K, Sumita K, Research S.M, Center D, Komukai-Toshiba-cho T.C, et al. (1994) Abstract generation based on rhetorical structure extraction. arXiv:cmp-lg/9411023

  2. Maredj A-E, Sadallah M, Hamouche L (2021) Une cinquième dimension pour les documents multimédia: La dimension annotation. Revue de l’Information Scientifique et Technique 25(2):12–20

    Google Scholar 

  3. Mann WC, Thompson SA (1988) Rhetorical structure theory: toward a functional theory of text organization. Text-Interdiscip. J. Study Discourse 8(3):243–281

    Article  Google Scholar 

  4. Hovy EH (1993) Automated discourse generation using discourse structure relations. Artif. Intell. 63(1–2):341–385

    Article  Google Scholar 

  5. Marcu D (1996) Building up rhetorical structure trees. In: Proceedings of the national conference on artificial intelligence, pp. 1069–1074

  6. Marcu D (2000) The theory and practice of discourse parsing and summarization. MIT Press, Cambridge

    Book  Google Scholar 

  7. Roisin C (1998) Authoring structured multimedia documents. In: SOFSEM’98: theory and practice of informatics: 25th conference on current trends in theory and practice of informatics Jasná, Slovakia, November 21–27, 1998 Proceedings 25, pp 222–239 . Springer

  8. André E, Müller J, Rist T (1996) The ppp persona: a multipurpose animated presentation agent. In: Proceedings of the workshop on advanced visual interfaces, pp 245–247

  9. Nack F, Hardman L (2001) Denotative and connotative semantics in hypermedia: proposal for a semiotic-aware architecture. New Rev Hypermedia Multimed 7(1):7–37

    Article  Google Scholar 

  10. Brusilovsky P (2001) Adaptive hypermedia. User Model. User Adapted Interact 11:87–110

    Article  Google Scholar 

  11. Geurts J, Bocconi S, Van Ossenbruggen J, Hardman L (2003) Towards ontology-driven discourse: from semantic graphs to multimedia presentations. In: The semantic web-ISWC 2003: second international semantic web conference, Sanibel Island, FL, USA, 2003. Proceedings 2, pp 597–612. Springer

  12. Yao W, He J, Huang G, Cao J, Zhang Y (2015) A graph-based model for context-aware recommendation using implicit feedback data. World Wide Web 18:1351–1371

    Article  Google Scholar 

  13. Haase F (2019) “Presentation” and “representation” of contents as principles of media convergence: a model of rhetorical narrativity of interactive multimedia design in mass communication with a case study of the digital edition of the new york times. Semiotica 2019(228): 1–29

  14. Hou S, Zhang S, Fei C (2020) Rhetorical structure theory: a comprehensive review of theory, parsing methods and applications. Expert Syst. Appl. 157:113421

    Article  Google Scholar 

  15. Taboada M, Mann WC (2006) Rhetorical structure theory: looking back and moving ahead. Discourse Stud. 8(3):423–459

    Article  Google Scholar 

  16. Joty S, Guzmán F, Màrquez L, Nakov P (2017) Discourse structure in machine translation evaluation. Computational Linguistics 43(4):683–722

    Article  MathSciNet  Google Scholar 

  17. Osman AH, Barukub OM (2020) Graph-based text representation and matching: a review of the state of the art and future challenges. IEEE Access 8:87562–87583

    Article  Google Scholar 

  18. Fei H, Zhang Y, Ren Y, Ji D (2021) A span-graph neural model for overlapping entity relation extraction in biomedical texts. Bioinformatics 37(11):1581–1589

    Article  CAS  PubMed  Google Scholar 

  19. Fei H, Wu S, Ren Y, Zhang M (2022) Matching structure for dual learning. In: International conference on machine learning, pp 6373–6391 . PMLR

  20. Fei H, Li F, Li B, Ji D (2021) Encoder-decoder based unified semantic role labeling with label-aware syntax. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 12794–12802

  21. Fei H, Wu S, Li J, Li B, Li F, Qin L, Liu Y, Teng C, Chua T-S (2022) Lasuie: unifying information extraction with latent adaptive structure-aware generative language model. Adv Neural Inf Process Syst 35:15460–15475

    Google Scholar 

  22. Khanday AMUD, Khan QR, Rabani ST (2021) Detecting textual propaganda using machine learning techniques. Baghdad Sci J 18(1):0199–0199

    Article  Google Scholar 

  23. Khanday AMUD, Wani MA, Rabani ST, Khan QR (2023) Hybrid approach for detecting propagandistic community and core node on social networks. Sustainability 15(2):1249

    Article  Google Scholar 

  24. Hochin T (2006) Graph-based data model for the content representation of multimedia data. In: Knowledge-based intelligent information and engineering systems: 10th international conference, KES 2006, Bournemouth, UK, 2006. Proceedings, Part II 10, pp 1182–1190 . Springer

  25. Zhang J, Zhu Y, Liu Q, Wu S, Wang S, Wang L (2021) Mining latent structures for multimedia recommendation. In: Proceedings of the 29th ACM international conference on multimedia, pp 3872–3880

  26. Wei Y, Wang X, He X, Nie L, Rui Y, Chua T-S (2021) Hierarchical user intent graph network for multimedia recommendation. IEEE Transactions on Multimedia 24:2701–2712

    Article  Google Scholar 

  27. Fei H, Ren Y, Ji D (2020) Boundaries and edges rethinking: an end-to-end neural model for overlapping entity relation extraction. Inf Process Manag 57(6):102311

    Article  Google Scholar 

  28. Wu S, Fei H, Ren Y, Donghong J, Jinyi L (2021) Learn from syntax: improving pair-wise aspect and opinion terms extraction with rich syntactic knowledge. arXiv preprint arXiv:2105.02520

  29. Hao F, Yafeng R, Yue Z, Donghong J, Xiaodan L (2021) Enriching contextualized language model from knowledge graph for biomedical information extraction. Brief Bioinform 22(3):110

    Article  Google Scholar 

  30. Maredj A-E, Sadallah M (2023) A set of rhetorical relationships for educational multimedia document. Revue de l’Information Scientifique et Technique 27(1):1–7

    Google Scholar 

  31. Bilasco IM, Gensel J, Villanova-Oliver M (2005) Stamp: a model for generating adaptable multimedia presentations. Multimed Tools Appl 25:361–375

    Article  Google Scholar 

  32. vanOssenbruggen JR, Cornelissen FJ, Geurts J, Rutledge L, Hardman L (2000) Cuypers: a semi-automatic hypermedia generation system. Information Systems [INS] (R 0025)

  33. Bosma W (2005) Query-based summarization using rhetorical structure theory. LOT Occas Ser 4:29–44

    Google Scholar 

  34. D’Armenio E (2022) The rhetorical dimension of images: identity building and management on social networks. Semiotica 2022(246):87–115

    Article  Google Scholar 

  35. Kjeldsen JE (2018) Visual rhetorical argumentation. Semiotica 2018(220):69–94

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DTISI, Research Center on Scientic and Technical Information CERIST, 05 Rue des 3 Frères Aissou, 16028, Ben Aknoun, Algiers, Algerie

    Azze-Eddine Maredj, Madjid Sadallah & Nourreddine Tonkin

Authors
  1. Azze-Eddine Maredj
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Madjid Sadallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nourreddine Tonkin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed equally to the conception, design, analysis, and interpretation of the research presented in this paper. Each author played a critical role in drafting and revising the manuscript for intellectual content, and all authors have read and approved the final version of the manuscript.

Corresponding author

Correspondence to Madjid Sadallah.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Maredj, AE., Sadallah, M. & Tonkin, N. Enhancing multimedia document modeling through extended orbit-based rhetorical structure: an approach to media weighting for importance determination. Knowl Inf Syst 66, 1683–1707 (2024). https://doi.org/10.1007/s10115-023-01984-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01984-6

Keywords

Search

Navigation