Abstract
In order to overcome the inefficiency and resource consumption of full-text search in relational databases, a light full-text search model with auxiliary cache is developed. Specially, we utilize the MySQL as the data storage layer and the Redis as the index cache layer. We first design a full-index cache mechanism by the Redis-based inverted indexes construction methods to augment the efficient memory processing capability of relational databases. In addition, an increment-index synchronization mechanism is implemented to fit the dynamic update of relation database. For hot data, an index update optimization mechanism is provided to guarantee the fast response and accuracy of full-text search. The proposed Redis-based auxiliary cache method has also been put into practical industrial applications and achieved promising results. Finally, we evaluate our method from index space occupation, time consumption and the accuracy of retrieval results. The experimental results show that the proposed model outperforms MySQL Full-Text method 2–3 times and surpasses ElasticSearch 12 times in space resource consumption.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Figa_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs13042-024-02160-0/MediaObjects/13042_2024_2160_Fig9_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
References
Jatana N, Puri S, Ahuja M, Kathuria I, Gosain D (2012) A survey and comparison of relational and non-relational database. Int J Eng Res Technol 1(6):1–5
Qtaish A, Alshammari MT (2019) A narrative review of storing and querying xml documents using relational database. J Inf Knowl Manag 18(04):1950048
Kadilierakis G Nikas C, Fafalios P, Papadakos P, Tzitzikas Y (2020) Elas4rdf: Multi-perspective triple-centered keyword search over rdf using elasticsearch. In: Semantic Web: ESWC 2020 Satellite, pp. 122–128
Čerešňák R, Kvet M (2019) Comparison of query performance in relational a non-relation databases. Transp Res Procedia 40:170–177
Jin D, Chen G, Hao W, Bin L (2020) Whole database retrieval method of general relational database based on lucene. In: Proceedings of the International Conference on Artificial Intelligence and Computer Applications, pp. 1277–1279
Safder I, Hassan S-U, Visvizi A, Noraset T, Nawaz R, Tuarob S (2020) Deep learning-based extraction of algorithmic metadata in full-text scholarly documents. Inf Process Manag 57(6):102269
Roy I, Velugoti VSBR, Chakrabarti S, De A (2022) Interpretable neural subgraph matching for graph retrieval. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 8115–8123
Wu X (2020) Analysis of data fuzzy retrieval method in embedded database (mysql). Comput Eng Softw 41(9):115–118
Akram S (2021) Exploiting intel optane persistent memory for full text search. In: Proceedings of the ACM SIGPLAN International Symposium on Memory Management, pp. 80–93
Chaitanya BSSK, Reddy DAK, Chandra BPSE, Krishna AB, Menon RR (2019) Full-text search using database index. In: Proceedings of the International Conference on Computing, Communication, Control and Automation, pp. 1–5
Gyorodi C, Gyorodi R, Pecherle G, Cornea GM (2010) Full-text search engine using mysql. Int J Comput Commun Control 5(5):735–743
Bast H, Buchhold B (2013) An index for efficient semantic full-text search. In: Proceedings of the ACM International Conference on Information & Knowledge Management, pp. 369–378
Iqbal W, Malik WI, Bukhari F, Almustafa KM, Nawaz Z (2021) Big data full-text search index minimization using text summarization. Inf Technol Control 50(2):375–389
Zamfir V-A, Carabas M, Carabas C, Tapus N (2019) Systems monitoring and big data analysis using the elasticsearch system. In: Proceedings of the International Conference on Control Systems and Computer Science, pp. 188–193
Sun J, Nie P, Xu L, Zhang H (2022) Design and implementation of analyzer management system based on elasticsearch. In: Proceedings of the International Conference on Web Information Systems and Applications, pp. 254–266
Zhan J, Mao J, Liu Y, Guo J, Zhang M, Ma S (2021) Jointly optimizing query encoder and product quantization to improve retrieval performance. In: Proceedings of the ACM International Conference on Information & Knowledge Management, pp. 2487–2496
Devins J, Tibshirani J, Lin J (2022) Aligning the research and practice of building search applications: Elasticsearch and pyserini. In: Proceedings of the ACM International Conference on Web Search and Data Mining, pp. 1573–1576
Amato G, Bolettieri P, Carrara F, Falchi F, Gennaro C (2018) Large-scale image retrieval with elasticsearch. In: Proceedings of the International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 925–928
Shah N, Willick D, Mago V (2022) A framework for social media data analytics using Elasticsearch and Kibana. Wireless Netw 28:1179–1187
Mackenzie J, Dai Z, Gallagher L, Callan J (2020) Efficiency implications of term weighting for passage retrieval. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1821–1824
Mallia A, Khattab O, Suel T, Tonellotto N (2021) Learning passage impacts for inverted indexes. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1723–1727
Shin Y, Ahn J, Im D (2022) Join optimization for inverted index technique on relational database management systems. Expert Syst Appl 198:116956
Zheng Q, Yang T, Kan Y, Tan X, Yang J, Jiang X (2021) On the analysis of cache invalidation with lru replacement. IEEE Trans Parallel Distrib Syst 33(3):654–666
Feng B, Tian A, Yu S, Li J, Zhou H, Zhang H (2022) Efficient cache consistency management for transient iot data in content-centric networking. IEEE Internet Things J 9(15):12931–12944
Pan C, Wang X, Luo Y, Wang Z (2021) Penalty-and locality-aware memory allocation in redis using enhanced aet. ACM Trans Storage 17(2):1–45
Su Q, Gao X, Zhang X, Wang Z (2022) A novel cache strategy leveraging redis with filters to speed up queries. In: Proceedings of the International Conference on High Performance Computing and Communication, vol. 12162, pp. 150–154
Matallah H, Belalem G, Bouamrane K (2020) Evaluation of nosql databases: Mongodb, cassandra, hbase, redis, couchbase, orientdb. Int J Softw Sci Comput Intell 12(4):71–91
Chen X, Wang F, Xu J, Zhu D, Tan P, Ma J (2020) A distributed cache system based on redis for high-speed railway catenary monitoring system. In: Proceedings of the Chinese Automation Congress, pp. 2048–2053
Zhang X, Wu P, Cai J, Wang K (2019) A contrastive study of Chinese text segmentation tools in marketing notification texts. In: Proceedings of the Journal of Physics: Conference Series, vol. 1302, p. 022010
Mallia A, Khattab O, Suel T, Tonellotto N (2021) Learning passage impacts for inverted indexes. In: Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1723–1727
Li P, Luo B, Zhu W, Xu H (2020) Cluster-based distributed dynamic cuckoo filter system for redis. Int J Parallel Emerg Distrib Syst 35(3):340–353
Gonçalves J, Matos M, Rodrigues R (2022) Sconekv: a scalable, strongly consistent key-value store. IEEE Trans Parallel Distrib Syst 33(12):4164–4175
Eddelbuettel D (2022) A brief introduction to redis. arXiv preprint arXiv:2203.06559
Siddiqui IF, Lee SU-J, Abbas A (2020) A novel knowledge-based battery drain reducer for smart meters. Intell Autom Soft Comput 26(1):107–119
Melnik S, Gubarev A, Long JJ, Romer G, Shivakumar S, Tolton M, Vassilakis T (2010) Dremel: interactive analysis of web-scale datasets. In: Proceedings of the VLDB Endowment, pp. 330–339
He J (2020) Research on personalized search based on elasticsearch. In: Proceedings of the International Conference on Civil Aviation Safety and Information Technology, pp. 572–575
Acknowledgements
This work was partially supported by grants from the National Science Foundation of China (No. 62176221), the National Social Science Fund of China (No. 20BMZ092).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Liao, X., Peng, L., Yang, T. et al. Redis-based full-text search extensions for relational databases. Int. J. Mach. Learn. & Cyber. 15, 4475–4491 (2024). https://doi.org/10.1007/s13042-024-02160-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-024-02160-0