Abstract
The rapid growth of digital information and the increasing complexity of user queries have made traditional search methods less effective in the context of business-related websites. This paper presents an innovative approach to improve the search experience across a variety of domains, particularly in the industrial sector, by integrating semantic search and conversational large language models such as GPT-3.5 into a domain-adaptive question-answering framework. Our proposed solution aims at complementing existing keyword-based approaches with the ability to capture entire questions or problems. By using all types of text, such as product manuals, documentation, advertisements, and other documents, all types of questions relevant to a website can be answered. These questions can be simple requests for product or domain knowledge, assistance in using a product, or more complex questions that may be relevant in determining the value of organizations as potential collaborators. We also introduce a mechanism for users to ask follow-up questions and to establish subject-specific communication with the search system. The results of our feasibility study show that the integration of semantic search and GPT-3.5 leads to significant improvements in the search experience, which could then translate into higher user satisfaction when querying the corporate portfolio. This research contributes to the ongoing development of advanced search technologies and has implications for a variety of industries seeking to unlock their hidden value.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
https://solr.apache.org, last accessed 2023-07-24.
- 2.
https://openai.com/blog/new-and-improved-embedding-model, last accessed 2023-07-24.
- 3.
https://streamlit.io, last accessed 2023-07-24.
- 4.
https://haystack.deepset.ai, last accessed 2023-07-24.
- 5.
https://platform.openai.com, last accessed 2023-07-24.
References
Almazrouei, E., et al.: Falcon-40B: an open large language model with state-of-the-art performance (2023)
Bast, H., Buchhold, B., Haussmann, E.: Semantic search on text and knowledge bases. Found. Trends® Inf. Retrieval 10(2–3), 119–271 (2016). https://doi.org/10.1561/1500000032
Brown, T., et al.: Language models are few-shot learners. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 1877–1901. Curran Associates, Inc. (2020). https://proceedings.neurips.cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
Cer, D., et al.: Universal sentence encoder for English. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 169–174. Association for Computational Linguistics, Brussels (2018). https://doi.org/10.18653/v1/D18-2029. https://aclanthology.org/D18-2029
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis (2019). https://doi.org/10.18653/v1/N19-1423. https://aclanthology.org/N19-1423
Hirschberg, J., Manning, C.D.: Advances in natural language processing. Science 349(6245), 261–266 (2015). https://doi.org/10.1126/science.aaa8685
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. CoRR abs/1907.11692 (2019). http://arxiv.org/abs/1907.11692
Ouyang, L., et al.: Training language models to follow instructions with human feedback. In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A. (eds.) Advances in Neural Information Processing Systems, vol. 35, pp. 27730–27744. Curran Associates, Inc. (2022). https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdf
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training. OpenAI Blog (2018)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1–67 (2020). http://jmlr.org/papers/v21/20-074.html
Reimers, N., Gurevych, I.: Sentence-BERT: sentence embeddings using Siamese BERT-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3982–3992. Association for Computational Linguistics, Hong Kong (2019). https://doi.org/10.18653/v1/D19-1410. https://aclanthology.org/D19-1410
Saini, B., Singh, V., Kumar, S.: Information retrieval models and searching methodologies: survey. Int. J. Adv. Found. Res. Sci. Eng. (IJAFRSE) 1, 20 (2014)
Sanh, V., Debut, L., Chaumond, J., Wolf, T.: DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR abs/1910.01108 (2019). http://arxiv.org/abs/1910.01108
Team, M.N.: Introducing MPT-7B: a new standard for open-source, commercially usable LLMs (2023). www.mosaicml.com/blog/mpt-7b. Accessed 24 July 2023
Touvron, H., et al.: LLaMA: open and efficient foundation language models (2023)
Vaswani, A., et al.: Attention is all you need. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017). https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf
Yang, Z., Dai, Z., Yang, Y., Carbonell, J.G., Salakhutdinov, R., Le, Q.V.: XLNet: generalized autoregressive pretraining for language understanding. In: Wallach, H.M., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E.B., Garnett, R. (eds.) Advances in Neural Information Processing Systems 32: NeurIPS 2019, Vancouver, BC, Canada, pp. 5754–5764 (2019)
Zhang, S., et al.: OPT: open pre-trained transformer language models (2022)
Acknowledgments
This work was co-funded by the German Federal Ministry of Education and Research under grants 13N16242 and 01IO2208E.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Maoro, F., Vehmeyer, B., Geierhos, M. (2024). Leveraging Semantic Search and LLMs for Domain-Adaptive Information Retrieval. In: Lopata, A., Gudonienė, D., Butkienė, R. (eds) Information and Software Technologies. ICIST 2023. Communications in Computer and Information Science, vol 1979. Springer, Cham. https://doi.org/10.1007/978-3-031-48981-5_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-48981-5_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48980-8
Online ISBN: 978-3-031-48981-5
eBook Packages: Computer ScienceComputer Science (R0)