ABSTRACT
Generative Adversarial Networks (GAN) have achieved great success in generating realistic synthetic data like images, tags, and sentences. We explore using GAN to generate bid keywords directly from query in sponsored search ads selection, especially for rare queries. Specifically, in the query expansion (query-keyword matching) scenario in search advertising, we train a sequence to sequence model as the generator to generate keywords, conditioned on the user query, and use a recurrent neural network model as the discriminator to play an adversarial game with the generator. By applying the trained generator, we can generate keywords directly from a given query, so that we can highly improve the effectiveness and efficiency of query-keyword matching based ads selection in search advertising. We trained the proposed model in the clicked query-keyword pair dataset from a commercial search advertising system. Evaluation results show that the generated keywords are more relevant to the given query compared with the baseline model and they have big potential to bring extra revenue improvement.
Supplemental Material
- Philip Bachman and Doina Precup. 2015. Data Generation as Sequential Decision Making. CoRR Vol. abs/1506.03504 (2015). arxiv:1506.03504 http://arxiv.org/abs/1506.03504 Google ScholarDigital Library
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. CoRR Vol. abs/1409.0473 (2014). arxiv:1409.0473 http://arxiv.org/abs/1409.0473Google Scholar
- Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks. CoRR Vol. abs/1506.03099 (2015). arxiv:1506.03099 http://arxiv.org/abs/1506.03099Google Scholar
- Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2016. Enriching word vectors with subword information. arXiv preprint arXiv:1607.04606 (2016).Google Scholar
- Andrei Broder, Peter Ciccolo, Evgeniy Gabrilovich, Vanja Josifovski, Donald Metzler, Lance Riedel, and Jeffrey Yuan. 2009. Online expansion of rare queries for sponsored search Proceedings of the 18th international conference on World wide web. ACM, 511--520. Google ScholarDigital Library
- Andrei Z. Broder, Peter Ciccolo, Marcus Fontoura, Evgeniy Gabrilovich, Vanja Josifovski, and Lance Riedel. 2008. Search advertising using web relevance feedback. Proceedings of the 17th ACM conference on Information and knowledge management. ACM, 1013--1022. Google ScholarDigital Library
- KyungHyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the Properties of Neural Machine Translation: Encoder-Decoder Approaches. CoRR Vol. abs/1409.1259 (2014). arxiv:1409.1259 http://arxiv.org/abs/1409.1259Google Scholar
- Yejin Choi, Marcus Fontoura, Evgeniy Gabrilovich, Vanja Josifovski, Mauricio Mediano, and Bo Pang. 2010. Using landing pages for sponsored search ad selection Proceedings of the 19th international conference on World wide web. ACM, 251--260. Google ScholarDigital Library
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research Vol. 12, Aug (2011), 2493--2537. Google ScholarDigital Library
- Ariel Fuxman, Panayiotis Tsaparas, Kannan Achan, and Rakesh Agrawal. 2008. Using the wisdom of the crowds for keyword generation Proceedings of the 17th international conference on World Wide Web. ACM, 61--70. Google ScholarDigital Library
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680. Google ScholarDigital Library
- Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and Harnessing Adversarial Examples. CoRR Vol. abs/1412.6572 (2014). arxiv:1412.6572 http://arxiv.org/abs/1412.6572Google Scholar
- Thore Graepel, Joaquin Quinonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. Omnipress.Google ScholarDigital Library
- Ishaan Gulrajani, Faruk Ahmed, Martín Arjovsky, Vincent Dumoulin, and Aaron C. Courville. 2017. Improved Training of Wasserstein GANs. CoRR Vol. abs/1704.00028 (2017). arxiv:1704.00028 http://arxiv.org/abs/1704.00028Google ScholarDigital Library
- Dustin Hillard, Stefan Schroedl, Eren Manavoglu, Hema Raghavan, and Chirs Leggetter. 2010. Improving ad relevance in sponsored search. In Proceedings of the third ACM international conference on Web search and data mining. ACM, 361--370. Google ScholarDigital Library
- Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical Reparameterization with Gumbel-Softmax. https://arxiv.org/abs/1611.01144Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR Vol. abs/1412.6980 (2014). arxiv:1412.6980 http://arxiv.org/abs/1412.6980Google Scholar
- Matt J. Kusner and José Miguel Hernández-Lobato. 2016. GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution. CoRR Vol. abs/1611.04051 (2016). arxiv:1611.04051 http://arxiv.org/abs/1611.04051Google Scholar
- H. Brendan McMahan, Gary Holt, David Sculley, Michael Young, Dietmar Ebner, Julian Grady, Lan Nie, Todd Phillips, Eugene Davydov, Daniel Golovin, et al. 2013. Ad click prediction: a view from the trenches. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1222--1230. Google ScholarDigital Library
- Tomávs Mikolov. 2012. Statistical language models based on neural networks. Presentation at Google, Mountain View, 2nd April (2012).Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).Google Scholar
- Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014).Google Scholar
- Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).Google Scholar
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532--1543.Google ScholarCross Ref
- Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence Level Training with Recurrent Neural Networks. CoRR Vol. abs/1511.06732 (2015). arxiv:1511.06732 http://arxiv.org/abs/1511.06732Google Scholar
- Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved Techniques for Training GANs. CoRR Vol. abs/1606.03498 (2016). arxiv:1606.03498 http://arxiv.org/abs/1606.03498Google Scholar
- Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2015. Minimum Risk Training for Neural Machine Translation. CoRR Vol. abs/1512.02433 (2015). arxiv:1512.02433 http://arxiv.org/abs/1512.02433Google Scholar
- Yelong Shen, Xiaodong He, Jianfeng Gao, Li Deng, and Grégoire Mesnil. 2014. A Latent Semantic Model with Convolutional-Pooling Structure for Information Retrieval Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management (CIKM '14). ACM, New York, NY, USA, 101--110. Google ScholarDigital Library
- Amanda Spink, Dietmar Wolfram, Major B. J. Jansen, and Tefko Saracevic. 2001. Searching the web: The public and their queries. Journal of the Association for Information Science and Technology, Vol. 52, 3 (2001), 226--234. Google ScholarDigital Library
- Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks Advances in neural information processing systems. 3104--3112. Google ScholarDigital Library
- Richard S. Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation Proceedings of the 12th International Conference on Neural Information Processing Systems (NIPS'99). MIT Press, Cambridge, MA, USA, 1057--1063. http://dl.acm.org/citation.cfm?id=3009657.3009806 Google ScholarDigital Library
- Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2016. Neural Machine Translation with Reconstruction. CoRR Vol. abs/1611.01874 (2016). arxiv:1611.01874 http://arxiv.org/abs/1611.01874Google Scholar
- Sam Wiseman and Alexander M. Rush. 2016. Sequence-to-Sequence Learning as Beam-Search Optimization. CoRR Vol. abs/1606.02960 (2016). arxiv:1606.02960 http://arxiv.org/abs/1606.02960Google Scholar
- Lijun Wu, Yingce Xia, Li Zhao, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2017. Adversarial Neural Machine Translation. CoRR Vol. abs/1704.06933 (2017). arxiv:1704.06933 http://arxiv.org/abs/1704.06933Google Scholar
- Zhuoran Xu, Xiangzhi Wang, and Yong Yu. 2011. Rare Query Expansion via Wikipedia for Sponsored Search. Knowledge Engineering and Management. Springer, 521--530.Google Scholar
- Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. AAAI. 2852--2858.Google Scholar
- Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, and Dimitris N. Metaxas. 2016. StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks. CoRR Vol. abs/1612.03242 (2016). arxiv:1612.03242 http://arxiv.org/abs/1612.03242Google Scholar
Index Terms
- Rare Query Expansion Through Generative Adversarial Networks in Search Advertising
Recommendations
An Empirical Analysis of Search Engine Advertising: Sponsored Search in Electronic Markets
The phenomenon of sponsored search advertising---where advertisers pay a fee to Internet search engines to be displayed alongside organic (nonsponsored) Web search results---is gaining ground as the largest source of revenues for search engines. Using a ...
An empirical analysis of sponsored search performance in search engine advertising
WSDM '08: Proceedings of the 2008 International Conference on Web Search and Data MiningThe phenomenon of sponsored search advertising - where advertisers pay a fee to Internet search engines to be displayed alongside organic (non-sponsored) web search results - is gaining ground as the largest source of revenues for search engines. ...
Competitive Poaching in Sponsored Search Advertising and Its Strategic Impact on Traditional Advertising
Traditional advertising, such as TV and print advertising, primarily builds awareness of a firm's product among consumers, whereas sponsored search advertising on a search engine can target consumers closer to making a purchase because they reveal their ...
Comments