short-paper

A Preliminary Exploration of Extractive Multi-Document Summarization in Hyperbolic Space

Authors:

Liping JingAuthors Info & Claims

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

Pages 4505 - 4509

https://doi.org/10.1145/3511808.3557538

Published: 17 October 2022 Publication History

Abstract

Summary matching is a recently proposed paradigm for extractive summarization. It aims to calculate similarities between candidate summaries and their corresponding document and extract summaries by ranking similarities. Due to natural languages often exhibiting the inherent hierarchical structures ingrained with complex syntax and semantics, the latent hierarchical structures between candidate summaries and their corresponding document should be considered when calculating the summary-document similarities. However, the above structural property is hard to model in the Euclidean space. Inspired by the above issues, we explore extractive summarization in the hyperbolic space and propose a new Hyperbolic Siamese Network for the matching-based extractive summarization (HyperSiameseNet). Specifically, HyperSiameseNet projects candidate summaries and their corresponding document representations from the Euclidean space to the Hyperbolic space and then models the summary-document similarities via the squared poincaré distance. Finally, the summary-document similarities are optimized by the margin-based triplet loss for extracting the final summary. The results on the Multi-News dataset have shown the superiority of our model HyperSiameseNet by comparing with the state-of-the-art baselines.

References

[1]

Jaime G. Carbonell and Jade Goldstein. 2017. The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries. SIGIR Forum, Vol. 51, 2 (2017), 209--210. https://doi.org/10.1145/3130348.3130369

Digital Library

[2]

Boli Chen, Yao Fu, Guangwei Xu, Pengjun Xie, Chuanqi Tan, Mosha Chen, and Liping Jing. 2021. Probing BERT in Hyperbolic Spaces. In International Conference on Learning Representations. https://openreview.net/forum?id=17VnwXYZyhH

[3]

Boli Chen, Xin Huang, Lin Xiao, Zixin Cai, and Liping Jing. 2020b. Hyperbolic Interaction Model for Hierarchical Multi-Label Classification. In AAAI. AAAI Press, 7496--7503. http://dblp.uni-trier.de/db/conf/aaai/aaai2020.html#ChenHXCJ20

[4]

Boli Chen, Xin Huang, Lin Xiao, and Liping Jing. 2020a. Hyperbolic Capsule Networks for Multi-Label Classification. In ACL, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 3115--3124. http://dblp.uni-trier.de/db/conf/acl/acl2020.html#ChenHXJ20

[5]

Shuyang Dai, Zhe Gan, Yu Cheng, Chenyang Tao, Lawrence Carin, and Jingjing Liu. 2020. APo-VAE: Text Generation in Hyperbolic Space. CoRR, Vol. abs/2005.00054 (2020). http://dblp.uni-trier.de/db/journals/corr/corr2005.html#abs-2005-00054

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT (1), Jill Burstein, Christy Doran, and Thamar Solorio (Eds.). Association for Computational Linguistics, 4171--4186. http://dblp.uni-trier.de/db/conf/naacl/naacl2019--1.html#DevlinCLT19

[7]

Gü nes Erkan and Dragomir R. Radev. 2011. LexRank: Graph-based Lexical Centrality as Salience in Text Summarization. CoRR, Vol. abs/1109.2128 (2011). showeprint[arXiv]1109.2128 http://arxiv.org/abs/1109.2128

[8]

Alexander R. Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir R. Radev. 2019. Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Anna Korhonen, David R. Traum, and Llu'i s Mà rquez (Eds.). Association for Computational Linguistics, 1074--1084. https://doi.org/10.18653/v1/p19--1102

[9]

Octavian-Eugen Ganea, Gary Bécigneul, and Thomas Hofmann. 2018. Hyperbolic Neural Networks. In NeurIPS, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 5350--5360. http://dblp.uni-trier.de/db/conf/nips/nips2018.html#GaneaBH18

[10]

Sebastian Gehrmann, Yuntian Deng, and Alexander M. Rush. 2018. Bottom-Up Abstractive Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii (Eds.). Association for Computational Linguistics, 4098--4109. https://doi.org/10.18653/v1/d18--1443

[11]

Caglar Gulcehre, Misha Denil, Mateusz Malinowski, Ali Razavi, Razvan Pascanu, Karl Moritz Hermann, Peter W. Battaglia, Victor Bapst, David Raposo, Adam Santoro, and Nando de Freitas. 2019. Hyperbolic Attention Networks. In ICLR (Poster). OpenReview.net. http://dblp.uni-trier.de/db/conf/iclr/iclr2019.html#GulcehreDMRPHBB19

[12]

Matthias Hamann. 2018. On the tree-likeness of hyperbolic spaces. In Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 164. 345--361.

[13]

C. Hopper and B. Andrews. 2011. The Ricci Flow in Riemannian Geometry. The Ricci flow in Riemannian geometry.

[14]

Valentin Khrulkov, Leyla Mirvakhabova, Evgeniya Ustinova, Ivan V. Oseledets, and Victor S. Lempitsky. 2020. Hyperbolic Image Embeddings. In CVPR. IEEE, 6417--6427. http://dblp.uni-trier.de/db/conf/cvpr/cvpr2020.html#KhrulkovMUOL20

[15]

Logan Lebanoff, Kaiqiang Song, and Fei Liu. 2018. Adapting the Neural Encoder-Decoder Framework from Single to Multi-Document Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ichi Tsujii (Eds.). Association for Computational Linguistics, 4131--4141. https://doi.org/10.18653/v1/d18--1446

[16]

C. Y. Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS). Barcelona, Spain.

[17]

Yang Liu. 2019. Fine-tune BERT for Extractive Summarization. CoRR, Vol. abs/1903.10318 (2019). http://dblp.uni-trier.de/db/journals/corr/corr1903.html#abs-1903--10318

[18]

Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing Order into Text. In EMNLP. ACL, 404--411. http://dblp.uni-trier.de/db/conf/emnlp/emnlp2004.html#MihalceaT04

[19]

Maximilian Nickel and Douwe Kiela. 2017. Poincaré Embeddings for Learning Hierarchical Representations. In NIPS. 6338--6347. http://dblp.uni-trier.de/db/conf/nips/nips2017.html#NickelK17

[20]

Christopher De Sa, Albert Gu, Christopher Ré, and Frederic Sala. 2018. Representation Tradeoffs for Hyperbolic Embeddings. CoRR, Vol. abs/1804.03329 (2018). arxiv: 1804.03329 http://arxiv.org/abs/1804.03329

[21]

Rik Sarkar. 2011. Low Distortion Delaunay Embedding of Trees in Hyperbolic Plane. In Graph Drawing (Lecture Notes in Computer Science, Vol. 7034), Marc J. van Kreveld and Bettina Speckmann (Eds.). Springer, 355--366. http://dblp.uni-trier.de/db/conf/gd/gd2011.html#Sarkar11

[22]

Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get To The Point: Summarization with Pointer-Generator Networks. In ACL (1), Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, 1073--1083. http://dblp.uni-trier.de/db/conf/acl/acl2017--1.html#SeeLM17

[23]

Mingyang Song, Yi Feng, and Liping Jing. 2022. Hyperbolic Relevance Matching for Neural Keyphrase Extraction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2022, Seattle, WA, United States, July 10--15, 2022, Marine Carpuat, Marie-Catherine de Marneffe, and Ivá n Vladimir Meza Ru'i z (Eds.). Association for Computational Linguistics, 5710--5720. https://doi.org/10.18653/v1/2022.naacl-main.419

[24]

Alexandru Tifrea, Gary Bécigneul, and Octavian-Eugen Ganea. 2019. Poincare Glove: Hyperbolic Word Embeddings. In ICLR (Poster). OpenReview.net. http://dblp.uni-trier.de/db/conf/iclr/iclr2019.html#TifreaBG19

[25]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998--6008. http://dblp.uni-trier.de/db/conf/nips/nips2017.html#VaswaniSPUJGKP17

[26]

Danqing Wang, Pengfei Liu, Yining Zheng, Xipeng Qiu, and Xuanjing Huang. 2020. Heterogeneous Graph Neural Networks for Extractive Document Summarization. CoRR, Vol. abs/2004.12393 (2020). showeprint[arXiv]2004.12393 https://arxiv.org/abs/2004.12393

[27]

Chao Zhao, Tenghao Huang, Somnath Basu Roy Chowdhury, Muthu Kumar Chandrasekaran, Kathleen R. McKeown, and Snigdha Chaturvedi. 2022. Read Top News First: A Document Reordering Approach for Multi-Document News Summarization. CoRR, Vol. abs/2203.10254 (2022). https://doi.org/10.48550/arXiv.2203.10254 showeprint[arXiv]2203.10254

[28]

Ming Zhong, Pengfei Liu, Yiran Chen, Danqing Wang, Xipeng Qiu, and Xuanjing Huang. 2020. Extractive Summarization as Text Matching. In ACL, Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.). Association for Computational Linguistics, 6197--6208. http://dblp.uni-trier.de/db/conf/acl/acl2020.html#ZhongLCWQH20

Cited By

Nielsen FOkamura K(2024)Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributionsInformation Geometry10.1007/s41884-024-00158-97:S2(943-989)Online publication date: 27-Nov-2024
https://doi.org/10.1007/s41884-024-00158-9
Song MLiu HJing LFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Diversity in Unsupervised Keyphrase Extraction with Determinantal Point ProcessProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615141(4294-4299)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615141
Song MFeng YJing L(2023)HISum: Hyperbolic Interaction Model for Extractive Multi-Document SummarizationProceedings of the ACM Web Conference 202310.1145/3543507.3583197(1427-1436)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583197
Show More Cited By

Index Terms

A Preliminary Exploration of Extractive Multi-Document Summarization in Hyperbolic Space
1. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing
      1. Information extraction
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Summarization

Recommendations

HISum: Hyperbolic Interaction Model for Extractive Multi-Document Summarization
WWW '23: Proceedings of the ACM Web Conference 2023

Extractive summarization helps provide a short description or a digest of news or other web texts. It enhances the reading experience of users, especially when they are reading on small displays (e.g., mobile phones). Matching-based methods are recently ...
Using topic themes for multi-document summarization

The problem of using topic representations for multidocument summarization (MDS) has received considerable attention recently. Several topic representations have been employed for producing informative and coherent summaries. In this article, we ...
Topic themes for multi-document summarization
SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval

The problem of using topic representations for multi-document summarization (MDS) has received considerable attention recently. In this paper, we describe five different topic representations and introduce a novel representation of topics based on topic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CIKM '22: Proceedings of the 31st ACM International Conference on Information & Knowledge Management

October 2022

5274 pages

ISBN:9781450392365

DOI:10.1145/3511808

General Chairs:
Mohammad Al Hasan
Indiana University Purdue University, Indianapolis, USA
,
Li Xiong
Emory University, Atlanta, USA

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

The National Science Foundation of China
The National Key Research and Development Program of China
The Beijing Natural Science Foundation
The Fundamental Research Funds for the Central Universities
The National Science Foundation of China

Conference

CIKM '22

Sponsor:

CIKM '22: The 31st ACM International Conference on Information and Knowledge Management

October 17 - 21, 2022

GA, Atlanta, USA

Acceptance Rates

CIKM '22 Paper Acceptance Rate 621 of 2,257 submissions, 28%;

Overall Acceptance Rate 1,861 of 8,427 submissions, 22%

Upcoming Conference

CIKM '25

Sponsor:
sigir
sigir

The 34th ACM International Conference on Information and Knowledge Management

November 10 - 14, 2025

Seoul , Republic of Korea

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
169
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nielsen FOkamura K(2024)Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributionsInformation Geometry10.1007/s41884-024-00158-97:S2(943-989)Online publication date: 27-Nov-2024
https://doi.org/10.1007/s41884-024-00158-9
Song MLiu HJing LFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Improving Diversity in Unsupervised Keyphrase Extraction with Determinantal Point ProcessProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615141(4294-4299)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615141
Song MFeng YJing L(2023)HISum: Hyperbolic Interaction Model for Extractive Multi-Document SummarizationProceedings of the ACM Web Conference 202310.1145/3543507.3583197(1427-1436)Online publication date: 30-Apr-2023
https://dl.acm.org/doi/10.1145/3543507.3583197
Li SXu J(2023)HierMDS: a hierarchical multi-document summarization model with global–local document dependenciesNeural Computing and Applications10.1007/s00521-023-08680-035:25(18553-18570)Online publication date: 26-Jun-2023
https://dl.acm.org/doi/10.1007/s00521-023-08680-0
Li XWang JJing L(2023)Interpretable Image Recognition by Screening Class-Specific and Class-Shared PrototypesArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44210-0_32(397-408)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44210-0_32

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten