Loading [MathJax]/extensions/MathMenu.js
Framework for Real-Time Parallel and Distributed Natural Language Processing | IEEE Conference Publication | IEEE Xplore

Framework for Real-Time Parallel and Distributed Natural Language Processing


Abstract:

In this paper, we present a new framework for parallel and distributed processing of real-time text streams capable for executing NLP-Natural Language Processing algorith...Show More

Abstract:

In this paper, we present a new framework for parallel and distributed processing of real-time text streams capable for executing NLP-Natural Language Processing algorithms. The focus is set on acceleration based on attention for building the topology, and not on the individual NLP algorithms. We elaborate the configuration of our specific use case and discuss the reduction of the time required for system configuration in order to use the benefits of virtualization and containers. Research hypothesis: We can process more text tuples per unit time using the newly developed framework for an algorithm that divides the sequential algorithm into smaller jobs and tasks including tokenization, part of speech tagging, stopwords, sentiment analysis, where each of these individual jobs are specific nodes in the Apache Storm-based topology. We have conducted an experimental proof-of-concept and found the optimal configuration confirming the validity of the hypothesis.
Date of Conference: 27 September 2021 - 01 October 2021
Date Added to IEEE Xplore: 15 November 2021
ISBN Information:
Electronic ISSN: 2623-8764
Conference Location: Opatija, Croatia

References

References is not available for this document.