Abstract
The levels-of-processing theory proposes that there are many ways to process and code information. The level of processing adopted will determine the quality of the representation used to store the information in the computer memory or storage. The levels-of-processing applied in information retrieval can be classified as follows: string processing, morphological processing, syntactic processing and semantic processing. These level-of-processing are imbedded into various models of information retrieval. Conventional information retrieval models, such as Boolean and vector space models rely on an extensive use of keywords, as independent strings, and their frequencies in storing and retrieving information. Thus string processing and morphological processing are mainly adopted in these models. It is believed that such an approach has reached its upper limit of retrieval effectiveness, and therefore, new approaches should be investigated for the development of future systems that will be more effective. With current advances in programming languages and techniques, natural language processing and understanding, and generally in the fields of artificial intelligence and cognitive science, there are now attempts made to include knowledge representation and linguistic processing into information retrieval systems. We also focus our research on the application of certain techniques on specific languages. Besides English, we focus the application of certain techniques especially on Malay. In this paper we will highlight some of the research done in the area of information retrieval at the various levels of processing, and also expound the current research we are doing and the future direction that we would like to undertake.
Keywords
- Information Retrieval
- Natural Language Processing
- Vector Space Model
- Character String
- Categorial Unification
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Othman, A.: Pengakar perkataan melayu untuk sistem capaian dokumen. MSc Thesis. National University of Malaysia (1993)
Allen, J.: Natural Language Understanding. The Benjamin/Cummings Publishing Company, Inc. (1987)
Davidson, L.: Retrieval of misspelled names in an airline’s passenger record system. Communications of the ACM 5, 169–171 (1962)
Ahmad, F., Yusoff, M., Mohd. T. Sembok, T.: Experiments with A Malay Stemming Algorithm. Journal of American Society of Information Science (1996)
Fox, E.A.: Characterization of Two New Experimental Collections in Computer and Information Science Containing Textual and Bibliographic Concepts, Technical Report 83–561, Department of Computer Science, Cornell University (1983)
Gadd, T.N.: ’Fisching Fore Werds’: phonetic retrieval of written text in information systems. Program 22, 222–237 (1988)
Gadd, T.N.: PHONIX: the algorithm. Program 24, 363–366 (1990)
Mizzaro, S.: Relevance: The Whole History. JASIS 48(9), 810–832 (1997)
Pollock, J.J., Zamora, A.: Correction and characterization of spelling errors in scientific and scholarly text. Journal of the American Society for Information Science 34, 51–58 (1983)
Pollock, J.J., Zamora, A.: Automatic spelling correction in scientific and scholarly text. Communications of the ACM 27, 358–368 (1984)
Porter, M.F.: An Algorithm for suffix stripping. Program 14(3), 130–137 (1980)
Rogers, H.J., Willett, P.: Searching for historical word forms in text database using spelling-correction methods: reverse error and phonetic coding methods. Journal of Documentation 47, 333–353 (1991)
Russell, R.C.: U.S. patent 1261167. Washington: United States Patent Office (1918)
Russell, R.C.: U.S. patent 1435663. Washington: United States Patent Office (1922)
Sembok, T.M.T., Willett, P.: Experiments with n-gram string-similarity measure on malay texts. Technical Report. Universiti Kebangsaan Malaysia (1995)
Sembok, T.M.T.: Application of Mathematical Functional Decomposition in Document Indexing, Prosiding: Pengintegrasian Technologi dalam Sains Matematik. Penang: USM (1999)
Bakar, Z.A., Sembok, T.M.T., Yusoff, M.: An Evaluation of Retrieval Effectiveness using Spelling-Correction and String Matching Methods on Malay Texts. Journal of American Society of Information Science, John Wiley (2000) (to appear)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sembok, T.M.T. (2003). Character Strings to Natural Language Processing in Information Retrieval. In: Sembok, T.M.T., Zaman, H.B., Chen, H., Urs, S.R., Myaeng, SH. (eds) Digital Libraries: Technology and Management of Indigenous Knowledge for Global Access. ICADL 2003. Lecture Notes in Computer Science, vol 2911. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24594-0_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-24594-0_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20608-8
Online ISBN: 978-3-540-24594-0
eBook Packages: Springer Book Archive