skip to main content
10.1145/3209978.3210184acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
tutorial

Generative Adversarial Nets for Information Retrieval: Fundamentals and Advances

Published: 27 June 2018 Publication History

Abstract

Generative adversarial nets (GANs) have been widely studied during the recent development of deep learning and unsupervised learning. With an adversarial training mechanism, GAN manages to train a generative model to fit the underlying unknown real data distribution under the guidance of the discriminative model estimating whether a data instance is real or generated. Such a framework is originally proposed for fitting continuous data distribution such as images, thus it is not straightforward to be directly applied to information retrieval scenarios where the data is mostly discrete, such as IDs, text and graphs. In this tutorial, we focus on discussing the GAN techniques and the variants on discrete data fitting in various information retrieval scenarios. (i) We introduce the fundamentals of GAN framework and its theoretic properties; (ii) we carefully study the promising solutions to extend GAN onto discrete data generation; (iii) we introduce IRGAN, the fundamental GAN framework of fitting single ID data distribution and the direct application on information retrieval; (iv) we further discuss the task of sequential discrete data generation tasks, e.g., text generation, and the corresponding GAN solutions; (v) we present the most recent work on graph/network data fitting with node embedding techniques by GANs. Meanwhile, we also introduce the relevant open-source platforms such as IRGAN and Texygen to help audience conduct research experiments on GANs in information retrieval. Finally, we conclude this tutorial with a comprehensive summarization and a prospect of further research directions for GANs in information retrieval.

References

[1]
Bahriye Akay, Ouguz Kaynar, and Ferhan Demirkoparan . 2017. Deep learning based recommender systems. In UBMK. IEEE, 645--648.
[2]
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer . 2015. Scheduled sampling for sequence prediction with recurrent neural networks Advances in Neural Information Processing Systems. 1171--1179.
[3]
Chris Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Greg Hullender . 2005. Learning to rank using gradient descent. In Proceedings of the 22nd international conference on Machine learning. ACM, 89--96.
[4]
Liwei Cai and William Yang Wang . 2017. KBGAN: Adversarial Learning for Knowledge Graph Embeddings. arXiv preprint arXiv:1711.04071 (2017).
[5]
Tong Che, Yanran Li, Ruixiang Zhang, R Devon Hjelm, Wenjie Li, Yangqiu Song, and Yoshua Bengio . 2017. Maximum-Likelihood Augmented Discrete Generative Adversarial Networks. arXiv preprint arXiv:1702.07983 (2017).
[6]
Paul Covington, Jay Adams, and Emre Sargin . 2016. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 191--198.
[7]
Ian Goodfellow . 2016. NIPS 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160 (2016).
[8]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio . 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.
[9]
Jiafeng Guo, Yixing Fan, Qingyao Ai, and W Bruce Croft . 2016. A deep relevance matching model for ad-hoc retrieval Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 55--64.
[10]
Jiaxian Guo, Sidi Lu, Han Cai, Weinan Zhang, Jun Wang, and Yong Yu . 2017. Long Text Generation via Adversarial Training with Leaked Information. arXiv preprint arXiv:1709.08624.
[11]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua . 2017. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web. International World Wide Web Conferences Steering Committee, 173--182.
[12]
Eric Jang, Shixiang Gu, and Ben Poole . 2017. Categorical reparameterization with gumbel-softmax. ICLR (2017).
[13]
Matt J Kusner and José Miguel Hernández-Lobato . 2016. Gans for sequences of discrete elements with the gumbel-softmax distribution. arXiv preprint arXiv:1611.04051 (2016).
[14]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton . 2015. Deep learning. nature Vol. 521, 7553 (2015), 436.
[15]
Kevin Lin, Dianqi Li, Xiaodong He, Zhengyou Zhang, and Ming-Ting Sun . 2017. Adversarial Ranking for Language Generation. arXiv preprint arXiv:1705.11001 (2017).
[16]
Tie-Yan Liu et almbox. . 2009. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval Vol. 3, 3 (2009), 225--331.
[17]
Sidi Lu, Lantao Yu, Weinan Zhang, and Yong Yu . 2018 a. CoT: Cooperative Training for Generative Modeling. arXiv preprint arXiv:1804.03782 (2018).
[18]
Sidi Lu, Yaoming Zhu, Weinan Zhang, Jun Wang, and Yong Yu . 2018 b. Neural Text Generation: Past, Present and Beyond. arXiv preprint arXiv:1803.07133 (2018).
[19]
Lin Ma, Zhengdong Lu, and Hang Li . 2016. Learning to Answer Questions from Image Using Convolutional Neural Network. AAAI, Vol. Vol. 3. 16.
[20]
Jay M Ponte and W Bruce Croft . 1998. A language modeling approach to information retrieval Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. ACM, 275--281.
[21]
Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil . 2014. Learning semantic representations using convolutional neural networks for web search. In Proceedings of the 23rd International Conference on World Wide Web. ACM, 373--374.
[22]
Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et almbox. . 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation. NIPS. 1057--1063.
[23]
Hongwei Wang, Jia Wang, Jialin Wang, Miao Zhao, Weinan Zhang, Fuzheng Zhang, Xing Xie, and Minyi Guo . 2017 a. GraphGAN: Graph Representation Learning with Generative Adversarial Nets. AAAI (2017).
[24]
Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang . 2017 b. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In SIGIR. ACM, 515--524.
[25]
Ying Wen, Weinan Zhang, Rui Luo, and Jun Wang . 2016. Learning text representation using recurrent convolutional neural network with highway layers. NeuIR Workshop (2016).
[26]
Ronald J Williams . 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning Vol. 8, 3--4 (1992), 229--256.
[27]
Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu . 2017. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. AAAI. 2852--2858.
[28]
Chengxiang Zhai and John Lafferty . 2004. A study of smoothing methods for language models applied to information retrieval. ACM Transactions on Information Systems (TOIS) Vol. 22, 2 (2004), 179--214.
[29]
Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu . 2018. Texygen: A Benchmarking Platform for Text Generation Models SIGIR.

Cited By

View all
  • (2024)Sample, Nudge and Rank: Exploiting Interpretable GAN Controls for Exploratory SearchProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645156(582-596)Online publication date: 18-Mar-2024
  • (2022)SigGAN: Adversarial Model for Learning Signed Relationships in NetworksACM Transactions on Knowledge Discovery from Data10.1145/353261017:1(1-20)Online publication date: 20-May-2022
  • (2022)Learning to rank method combining multi-head self-attention with conditional generative adversarial netsArray10.1016/j.array.2022.10020515(100205)Online publication date: Sep-2022
  • Show More Cited By

Index Terms

  1. Generative Adversarial Nets for Information Retrieval: Fundamentals and Advances

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval
      June 2018
      1509 pages
      ISBN:9781450356572
      DOI:10.1145/3209978
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 June 2018

      Check for updates

      Author Tags

      1. deep learning
      2. discrete data
      3. generative adversarial nets
      4. information retrieval
      5. network embedding
      6. reinforcement learning
      7. text generation

      Qualifiers

      • Tutorial

      Funding Sources

      Conference

      SIGIR '18
      Sponsor:

      Acceptance Rates

      SIGIR '18 Paper Acceptance Rate 86 of 409 submissions, 21%;
      Overall Acceptance Rate 792 of 3,983 submissions, 20%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)54
      • Downloads (Last 6 weeks)5
      Reflects downloads up to 14 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Sample, Nudge and Rank: Exploiting Interpretable GAN Controls for Exploratory SearchProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645156(582-596)Online publication date: 18-Mar-2024
      • (2022)SigGAN: Adversarial Model for Learning Signed Relationships in NetworksACM Transactions on Knowledge Discovery from Data10.1145/353261017:1(1-20)Online publication date: 20-May-2022
      • (2022)Learning to rank method combining multi-head self-attention with conditional generative adversarial netsArray10.1016/j.array.2022.10020515(100205)Online publication date: Sep-2022
      • (2021)Generative Adversarial Networks (GANs)ACM Computing Surveys10.1145/344637454:3(1-42)Online publication date: 8-May-2021
      • (2021)Multimodal Autoencoder Predicts fNIRS Resting State From EEG SignalsNeuroinformatics10.1007/s12021-021-09538-320:3(537-558)Online publication date: 10-Aug-2021
      • (2020)Generative Attribute Manipulation Scheme for Flexible Fashion SearchProceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3397271.3401150(941-950)Online publication date: 25-Jul-2020
      • (2020)Using Generative Adversarial Networks for Relevance Evaluation of Search Engine Results2020 IEEE East-West Design & Test Symposium (EWDTS)10.1109/EWDTS50664.2020.9224840(1-7)Online publication date: Sep-2020
      • (2020)Recommender Systems Based on Generative Adversarial Networks: A Problem-Driven PerspectiveInformation Sciences10.1016/j.ins.2020.09.013Online publication date: Sep-2020

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media