Abstract
Corpus analysis is one of the most powerful methods in text mining, data discovery, and finding relationships among documents. In linguistics, a corpus (plural corpora) is a large and structured set of texts which should to be classified by artificial intelligence systems. The performance of conventional text classifiers on corpora is usually unsatisfying. In this paper, a novel text classifier for corpus analysis is proposed by using advanced numerical unconstrained nonlinear optimization in collaboration with neural networks. The proposed approach, the relaxed conjugate gradient (RCG) trained artificial neural network, classifies each document using n-gram token filter by TF score multiplied by its IDF score. The proposed updating formula for training of neural networks combines the good numerical performance of Polak–Ribière technique and the wonderful global convergence properties of Fletcher–Reeves method and also it inherits some adaption from Hestenes–Stiefel, and Dai–Yuan conjugate gradient updating procedures by using the relaxation equation. The our proposed algorithm was evaluated on verses of Holy Quran and its outcomes were compared with results of its competitors such as the classical gradient descent algorithm, the modified quickprop algorithm, the conjugate gradient algorithm with Hestenes–Stiefel update, the conjugate gradient algorithm with Polak–Ribiere update, the conjugate gradient algorithm with Fletcher–Reeves updates, the scaled conjugate gradient algorithm, the variable memory Broyden–Fletcher–Goldfarb–Shanno update, and smoothed regularized conjugate gradient method. Based on these experiments, the proposed RCG is able to accurately classify text corpus with low computational cost.

Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Quran Databases. http://qurandatabase.org/. Accessed 23 June 2018
Nasr MB, Chtourou M (2006) A hybrid training algorithm for feedforward neural networks. Neural Process Lett 24(2):107–117
Cheng Y (2017) Backpropagation for fully connected cascade networks. Neural Process Lett 46(1):293–311
Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer, Berlin
Bandurski K, Kwedlo W (2010) A Lamarckian hybrid of differential evolution and conjugate gradients for neural network training. Neural Process Lett 32(1):31–44
Polak E, Ribiere G (1969) Note sur la convergence de méthodes de directions conjuguées. ESAIM Math Model Numer Anal 3(R1):35–43
Fletcher R, Reeves CM (1964) Function minimization by conjugate gradients. Comput J 7(2):149–154
Hestenes MR, Stiefel E (1953) Methods of conjugate gradients for solving linear systems. J Res Natl Bur Stand 49:409–436
Dai YH, Yuan Y (1999) A nonlinear conjugate gradient method with a strong global convergence property. SIAM J Optim 10(1):177–182
Watts FN, Turner L (2014) Evolution, religion, and cognitive science: critical and constructive essays. Oxford University Press, Oxford
Repko AR, Szostak R (2016) Interdisciplinary research: process and theory. SAGE Publications, Beverley Hills
Fogel D (2005) Evolutionary computation: toward a new philosophy of machine intelligence. Wiley, New York
Verma M (2017) Lexical analysis of religious texts using text mining. Int J Comput Appl 168(8):39–45
Dukes K (2009) The Quranic Arabic Corpus. http://corpus.quran.com/. Accessed 23 June 2018
Abdul-Baqi S (2012) Text mining of the Quran. http://textminingthequran.com/. Accessed 23 June 2018
Salton G, Wong A, Yang CS (1975) A vector space model for automatic indexing. Commun ACM 18(11):613–620
Read J, Reutemann P, Pfahringer B, Holmes G (2016) MEKA: a multi-label/multi-target extension to WEKA. J Mach Learn Res 17(21):1–5
Frank E, Hall MH, Witten IH (2016) The WEKA workbench. Online appendix for “data mining: practical machine learning tools and techniques”, 4th edn. Morgan Kaufmann, Los Altos
Tsoumakas G, Katakis I, Vlahavas I (2010) Mining multi-label data. In: Maimon O, Rokach L (eds) Data mining and knowledge discovery handbook, 2nd edn. Springer, Berlin
Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830
A multi-output/multi-label and stream data framework. https://scikit-multiflow.github.io/. Accessed 12 May 2018
Bifet A, Holmes G, Kirkby R, Pfahringer B (2010) MOA: massive online analysis. J Mach Learn Res 11:1601–1604
Popa C (2018) Learning algorithms for quaternion-valued neural networks. Neural Process Lett 47(3):949–973
Vrahatis MN, Magoulas GD, Plagianakos VP (2000) Globally convergent modification of the quickprop method. Neural Process Lett 12:159–169
Falas T, Stafylopatis A (2005) Implementing temporal-difference learning with the scaled conjugate gradient algorithm. Neural Process Lett 22(3):361–375
McLoone S, Irwin G (1999) A variable memory quasi-newton training algorithm. Neural Process Lett 9(1):77–89
Li W, Liu Y, Yang J et al (2018) A new conjugate gradient method with smoothing L1/2 regularization based on a modified secant equation for training neural networks. Neural Process Lett 48(2):955–978
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Borhani, M. Corpus Analysis Using Relaxed Conjugate Gradient Neural Network Training Algorithm. Neural Process Lett 50, 839–849 (2019). https://doi.org/10.1007/s11063-018-9948-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-018-9948-8