Skip to main content

Advertisement

Springer Nature Link
Log in

A survey on syntactic processing techniques

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Computational syntactic processing is a fundamental technique in natural language processing. It normally serves as a pre-processing method to transform natural language into structured and normalized texts, yielding syntactic features for downstream task learning. In this work, we propose a systematic survey of low-level syntactic processing techniques, namely: microtext normalization, sentence boundary disambiguation, part-of-speech tagging, text chunking, and lemmatization. We summarize and categorize widely used methods in the aforementioned syntactic analysis tasks, investigate the challenges, and yield possible research directions to overcome the challenges in future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://www.wiktionary.org/

  2. http://jazzy.sourceforge.net/

  3. http://www.nist.gov/speech/tests/rt/rt2003/fall/

References

  • Abney S, Schapire RE, Singer Y (1999) Boosting applied to tagging and pp attachment. In: 1999 joint SIGDAT conference on empirical methods in natural language processing and very large corpora

  • Agarwal N, Ford KH, Shneider M (2005) Sentence boundary detection using a maxEnt classifier. In: Proceedings of MISC, pp 1–6

  • Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66

    Google Scholar 

  • Akbik A, Blythe D, Vollgraf R (2018) Contextual string embeddings for sequence labeling. In: Proceedings of the 27th international conference on computational linguistics, pp 1638–1649

  • Akhmetov I, Pak A, Ualiyeva I, Gelbukh A (2020) Highly language-independent word lemmatization using a machine-learning classifier. Computación y Sistemas 24(3):1353

    Google Scholar 

  • Altinok D (2018) An ontology-based dialogue management system for banking and finance dialogue systems. arXiv preprint arXiv:1804.04838

  • Altun Y, Tsochantaridis I, Hofmann T (2003) Hidden markov support vector machines. In: Proceedings of the 20th international conference on machine learning (ICML-03), pp 3–10

  • Alva P, Hegde V (2016) Hidden Markov model for POS tagging in word sense disambiguation. In: 2016 international conference on computation system and information technology for sustainable solutions (CSITSS). IEEE, pp 279–284

  • Ando RK, Zhang T (2005) A framework for learning predictive structures from multiple tasks and unlabeled data. J Mach Learn Res 6(Nov):1817–1853

  • Arakelyan G, Hambardzumyan K, Khachatrian H (2018) Towards jointud: part-of-speech tagging and lemmatization using recurrent neural networks. arXiv preprint arXiv:1809.03211

  • Asghar MZ, Khan A, Ahmad S, Kundi FM (2014) A review of feature extraction in sentiment analysis. J Basic Appl Sci Res 4(3):181–186

    Google Scholar 

  • Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  • Balakrishnan V, Lloyd-Yemoh E (2014) Stemming and lemmatization: a comparison of retrieval performances. Lecture notes on software engineering, vol 2

  • Baldwin T, de Marneffe MC, Han B, Kim YB, Ritter A, Xu W (2015). Shared tasks of the 2015 workshop on noisy user-generated text: Twitter lexical normalization and named entity recognition. In: Proceedings of the workshop on noisy user-generated text, Beijing, China. Association for Computational Linguistics, pp 126–135

  • Bang J, Noh H, Kim Y, Lee GG (2015) Example-based chat-oriented dialogue system with personalized long-term memory. In: 2015 international conference on big data and smart computing (BIGCOMP). IEEE, pp 238–243

  • Barteld F, Schröder I, Zinsmeister H (2016) Dealing with word-internal modification and spelling variation in data-driven lemmatization. In: Proceedings of the 10th SIGHUM workshop on language technology for cultural heritage, social sciences, and humanities, pp 52–62

  • Bartlett S, Kondrak G, Cherry C (2008) Automatic syllabification with structured SVMs for letter-to-phoneme conversion. In: Proceedings of ACL-08: HLT, pp 568–576

  • Beider A (2008) Beider-morse phonetic matching: an alternative to soundex with fewer false hits. Avotaynu 24(2):12

    Google Scholar 

  • Belinkov Y, Màrquez L, Sajjad H, Durrani N, Dalvi F, Glass J (2018) Evaluating layers of representation in neural machine translation on part-of-speech and semantic tagging tasks. arXiv preprint arXiv:1801.07772

  • Bergmanis T, Goldwater S (2018) Context sensitive neural lemmatization with Lematus. In: Proceedings of the 2018 conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol 1 (Long Papers), pp 1391–1400

  • Bergmanis T, Goldwater S (2019) Training data augmentation for context-sensitive neural lemmatization using inflection tables and raw text. arXiv preprint arXiv:1904.01464

  • Biemann C (2006) Unsupervised part-of-speech tagging employing efficient graph clustering. In: Proceedings of the COLING/ACL 2006 student research workshop, pp 7–12

  • Böhmová A, Hajic J, Hajicová E, Hladká B, Abeillé A (2003) The prague dependency treebank: three-level annotation scenario. In: Treebanks: building and using parsed corpora, vol 20, pp 103–127

  • Bojanowski P, Grave E, Joulin A, Mikolov T (2016), 07. Enriching word vectors with subword information. Trans Assoc Comput Linguist. https://doi.org/10.1162/tacl_a_00051

  • Bontcheva K, Derczynski L, Funk A, Greenwood MA, Maynard D, Aswani N (2013) Twitie: An open-source information extraction pipeline for microblog text. Proceedings of the international conference recent advances in natural language processing RANLP 2013:83–90

    Google Scholar 

  • Boudin F, Huet S, Torres-Moreno JM (2011) A graph-based approach to cross-language multi-document summarization. Polibits 43:113–118

    Google Scholar 

  • Bouma G (2003) Finite state methods for hyphenation. Nat Lang Eng 9:5–20. https://doi.org/10.1017/S1351324903003073

    Article  Google Scholar 

  • Brants T (2000) TnT-a statistical part-of-speech tagger. arXiv preprint cs/0003055

  • Brill E (1995) Transformation-based error-driven learning and natural language processing: a case study in part-of-speech tagging. Comput Linguist 21(4):543–565

    Google Scholar 

  • Brill E, Wu J (1998) Classifier combination for improved lexical disambiguation. In: 36th annual meeting of the Association for Computational Linguistics and 17th international conference on computational linguistics, vol 1, pp 191–195

  • Brody S, Diakopoulos N (2011) Cooooooooooooooollllllllllllll!!!!!!!!!!!!!! using word lengthening to detect sentiment in microblogs. In: Proceedings of the 2011 conference on empirical methods in natural language processing, pp 562–570

  • Camastra F, Razi G (2020) Italian text categorization with lemmatization and support vector machines. In: Neural approaches to dynamics of signal exchanges. Springer, pp 47–54

  • Cambria E, Poria S, Gelbukh A, Thelwall M (2017) Sentiment analysis is a big suitcase. IEEE Intell Syst 32(6):74–80. https://doi.org/10.1109/MIS.2017.4531228

    Article  Google Scholar 

  • Cambria E, Liu Q, Decherchi S, Xing F, Kwok K (2022) Senticnet 7: a commonsense-based neurosymbolic AI framework for explainable sentiment analysis. Proceedings of LREC 2022:3829–3839

    Google Scholar 

  • Cappé O, Godsill SJ, Moulines E (2007) An overview of existing methods and recent advances in sequential monte carlo. Proc IEEE 95(5):899–924

    Google Scholar 

  • Cebron N, Berthold MR (2009) Active learning for object classification: from exploration to exploitation. Data Min Knowl Disc 18(2):283–299

    MathSciNet  Google Scholar 

  • Celano GG (2020) A gradient boosting-Seq2Seq system for Latin POS tagging and lemmatization. In: Proceedings of LT4HALA 2020-1st workshop on language technologies for historical and ancient languages, pp 119–123

  • Chakrabarty A, Pandit OA, Garain U (2017) Context sensitive lemmatization using two successive bidirectional gated recurrent networks. In: Proceedings of the 55th annual meeting of the Association for Computational Linguistics (vol 1: Long Papers), pp 1481–1491

  • Chakrabarty A, Chaturvedi A, Garain U (2019) CNN-based context sensitive lemmatization. In: Proceedings of the ACM India joint international conference on data science and management of data, CoDS-COMAD ’19, New York, NY, USA. Association for Computing Machinery, pp 334–337

  • Chen T, Kan MY (2012) Creating a live, public short message service corpus: the nus sms corpus. Lang Resour Eval. https://doi.org/10.1007/s10579-012-9197-9

    Article  Google Scholar 

  • Chen D, Manning CD (2014) A fast and accurate dependency parser using neural networks. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 740–750

  • Chiu JP, Nichols E (2016) Named entity recognition with bidirectional LSTM-CNNs. Trans Assoc Comput Linguist 4:357–370

    Google Scholar 

  • Choudhury M, Saraf R, Jain V, Mukherjee A, Sarkar S, Basu A (2007) Investigation and modeling of the structure of texting language. IJDAR 10(3):157–174. https://doi.org/10.1007/s10032-007-0054-0

    Article  Google Scholar 

  • Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint arXiv:1409.1259

  • Chrupała G (2006) Simple data-driven context-sensitive lemmatization. del Leng, Natural, Proces, p 37

    Google Scholar 

  • Chrupala G (2010) Morfette: a tool for supervised learning of morphology

  • Chrupała G (2014) Normalizing tweets with edit scripts and recurrent neural embeddings. In: Proceedings of the 52nd annual meeting of the Association for Computational Linguistics (vol 2: Short Papers), pp 680–686

  • Chrupała G, Dinu G, Genabith J (2008), 01. Learning morphology with Morfette. In: Chrupała G, Dinu G, van Genabith J (2008) Learning morphology with Morfette. In: LREC 2008 - sixth international conference on language resources and evaluation, 28–30 May 2008, Marrakech, Morocco

  • Church KW (1988) A stochastic parts program and noun phrase parser for unrestricted text. Second conference on applied natural language processing. Austin, Texas, USA. Association for Computational Linguistics, pp 136–143

    Google Scholar 

  • Civit M, Martí MA (2004) Building cast3lb: a Spanish treebank. Res Lang Comput 2(4):549–574

    Google Scholar 

  • Clark A (2003) Combining distributional and morphological information for part of speech induction. In: Proceedings of the tenth conference on European chapter of the Association for Computational Linguistics—vol 1, EACL ’03, USA. Association for Computational Linguistics, pp 59–66

  • Clark S, Curran JR, Osborne M (2003) Bootstrapping POS-taggers using unlabelled data. Proceedings of the seventh conference on natural language learning at HLT-NAACL 2003:49–55

    Google Scholar 

  • Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J Artif Intell Res 4:129–145

    MATH  Google Scholar 

  • Collins M (2002) Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In: Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP 2002). Association for Computational Linguistics, pp 1–8

  • Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537

    MATH  Google Scholar 

  • Compton P, Edwards G, Kang B, Lazarus L, Malor R, Preston P, Srinivasan A (1992) Ripple down rules: turning knowledge acquisition into knowledge maintenance. Artif Intell Med 4(6):463–475

    Google Scholar 

  • Councill I, McDonald R, Velikovich L (2010) What’s great and what’s not: learning to classify the scope of negation for improved sentiment analysis. In: Proceedings of the workshop on negation and speculation in natural language processing, pp 51–59

  • Curran JR, Clark S (2003) Investigating GIS and smoothing for maximum entropy taggers. In: 10th conference of the European chapter of the Association for Computational Linguistics

  • Daelemans W, Van den Bosch A, Weijters T (1997) Igtree: using trees for compression and classification in lazy learning algorithms. In: Lazy learning. Springer, pp 407–423

  • Daelemans W, Buchholz S, Veenstra J (1999a) Memory-based shallow parsing. arXiv preprint cs/9906005

  • Daelemans W, Zavrel J, Berck P, Gillis S (1999b) MBT: A memory-based part of speech tagger-generator. In: Fourth workshop on very large Corpora: 1996; Copenhagen, Denmark

  • Daelemans W, Zavrel J, van der Sloot K, van den Bosch A (2003) Timbl: Tilburg memory based learner, version 5.0, reference guide. Research Group Technical Report Series 3

  • Daelemans W, Zavrel J, Van Der Sloot K, Van den Bosch A (2004) Timbl: Tilburg memory-based learner. Tilburg University

  • Daelemans W, Groenewald HJ, van Huyssteen GB (2009) Prototype-based active learning for lemmatization. In: Proceedings of the international conference RANLP-2009, pp 65–70

  • Dai HJ, Lai PT, Chang YC, Tsai RTH (2015) Enhancing of chemical compound and drug name recognition using representative tag scheme and fine-grained tokenization. J Cheminform 7(1):1–10

    Google Scholar 

  • Darwish K, Mubarak H, Abdelali A, Eldesouki M, Samih Y, Alharbi R, Attia M, Magdy W, Kallmeyer L (2018) Multi-dialect Arabic POS tagging: a CRF approach. In: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018)

  • Dauphin YN, Fan A, Auli M, Grangier D (2017) Language modeling with gated convolutional networks. In: International conference on machine learning. PMLR, pp 933–941

  • Dereza O (2018) Lemmatization for ancient languages: rules or neural networks? In: Ustalov D, Filchenkov A, Pivovarova L, Žižka J (eds) Artificial intelligence and natural language. Springer, Cham, pp 35–47

    Google Scholar 

  • Desai N, Narvekar M (2015) Normalization of noisy text data. Procedia Comput Sci 45:127–132. https://doi.org/10.1016/j.procs.2015.03.104

    Article  Google Scholar 

  • Devlin J, Chang M, Lee K, Toutanova K (2018) BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs/1810.04805. arXiv:1810.04805

  • Dietterich TG, Bakiri G (1994) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    MATH  Google Scholar 

  • Dipper S, Lüdeling A, Reznicek M (2013) Nosta-d: a corpus of german non-standard varieties. Non-standard data sources in corpus-based research 5:69–76

    Google Scholar 

  • Dos Santos C, Zadrozny B (2014a) Learning character-level representations for part-of-speech tagging. In: International conference on machine learning. PMLR, pp 1818–1826

  • Dos Santos C, Zadrozny B (2014b) Learning character-level representations for part-of-speech tagging. In: International conference on machine learning. PMLR, pp 1818–1826

  • Dozat T, Manning CD (2016) Deep biaffine attention for neural dependency parsing. arXiv preprint arXiv:1611.01734

  • Dreyer M, Smith J, Eisner J (2008) Latent-variable modeling of string transductions with finite-state methods. In: Proceedings of the 2008 conference on empirical methods in natural language processing, pp 1080–1089

  • Dunlop M, Crossan A (2000) 08) Predictive text entry methods for mobile phones. Pers Technol. https://doi.org/10.1007/BF01324120

    Article  Google Scholar 

  • Dyer C, Ballesteros M, Ling W, Matthews A, Smith NA (2015) Transition-based dependency parsing with stack long short-term memory. arXiv preprint arXiv:1505.08075

  • Elman JL (1991) Distributed representations, simple recurrent networks, and grammatical structure. Mach Learn 7(2):195–225

    Google Scholar 

  • Erjavec T (1998) The multext-east Slovene lexicon. In: Proceedings of the 7th electrotechnical conference ERK, vol B, pp 189–192

  • Erjavec T, Džeroski S, (2004) Machine learning of morphosyntactic structure: lemmatizing unknown slovene words. Appl Artif Intell 18(1):17–41

    Google Scholar 

  • Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ (2008) Liblinear: a library for large linear classification. J Mach Learn Res 9:1871–1874

    MATH  Google Scholar 

  • Fatima M, Mueller MC (2019) HITS-SBD at the FinSBD task: machine learning vs. rule-based sentence boundary detection. In: Proceedings of the First workshop on financial technology and natural language processing, pp 115–121

  • Feng X, Feng Z, Zhao W, Zou N, Qin B, Liu T (2019) Improved neural machine translation with pos-tagging through joint decoding. In: International conference on artificial intelligence for communications and networks. Springer, pp 159–166

  • Forsyth EN (2007) Improving automated lexical and discourse analysis of online chat dialog. Technical report, NAVAL POSTGRADUATE SCHOOL MONTEREY CA

    Google Scholar 

  • Fossati D, Di Eugenio B (2008) I saw tree trees in the park: How to correct real-word spelling mistakes. In: LREC, pp 2008

  • Francis WN, Kucera H (1979) Brown corpus manual: manual of information to accompany a standard corpus of present-day edited american english for use with digital computers. Brown University, Providence, RI, USA

    Google Scholar 

  • Freitag D, McCallum A (2000) Information extraction with hmm structures learned by stochastic optimization. AAAI/IAAI 2000:584–589

    Google Scholar 

  • Gallay, L, Šimko M (2016) Utilizing vector models for automatic text lemmatization. In: International conference on current trends in theory and practice of informatics. Springer, pp 532–543

  • Ge M, Mao R, Cambria E (2022) Explainable metaphor identification inspired by conceptual metaphor theory. In: Proceedings of the 36th AAAI conference on artificial intelligence, pp 10681–10689

  • Gesmundo A, Samardzic T (2012) Lemmatisation as a tagging task. In: Proceedings of the 50th annual meeting of the Association for Computational Linguistics (vol 2: Short Papers), pp 368–372

  • Gillick D (2009) Sentence boundary detection and the problem with the US. In: Proceedings of human language technologies: the 2009 annual conference of the North American chapter of the Association for Computational Linguistics, companion volume: Short Papers, pp 241–244

  • Giménez J, Marquez L (2004a) Fast and accurate part-of-speech tagging: the SVM approach revisited. Rec Adv Nat Lang Process III:153–162

    Google Scholar 

  • Giménez J, Màrquez L (2004b) SVMTool: A general POS tagger generator based on support vector machines. In: Proceedings of the fourth international conference on language resources and evaluation (LREC’04), Lisbon, Portugal. European Language Resources Association (ELRA)

  • Gimpel K, Schneider N, O’Connor B, Das D, Mills D, Eisenstein J, Heilman M, Yogatama D, Flanigan J, Smith NA (2010) Part-of-speech tagging for twitter: annotation, features, and experiments. Technical report, Carnegie-Mellon Univ Pittsburgh Pa School of Computer Science

  • Gotoh Y, Renals S (2000) Sentence boundary detection in broadcast speech transcripts. In: ASR2000-automatic speech recognition: challenges for the new Millenium ISCA Tutorial and Research Workshop (ITRW)

  • Graves A, Mohamed Ar, Hinton G (2013) Speech recognition with deep recurrent neural networks. In: 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 6645–6649

  • Grefenstette G, Tapanainen P (1994) What is a word, what is a sentence?: Problems of tokenisation. Report, Grenoble Laboratory

    Google Scholar 

  • Griffis D, Shivade C, Fosler-Lussier E (2016) Lai AM (2016) A quantitative and qualitative evaluation of sentence boundary detection for the clinical domain. AMIA Jt Summits on Transl Sci Proc 27570656:88–97

    Google Scholar 

  • Grove AJ, Roth D (2001) Linear concepts and hidden variables. Mach Learn 42(1–2):123–141

    MATH  Google Scholar 

  • Gui T, Zhang Q, Huang H, Peng M, Huang X (2017), September. Part-of-speech tagging for twitter with adversarial neural networks. In: Proceedings of the 2017 conference on empirical methods in natural language processing, Copenhagen, Denmark. Association for Computational Linguistics, pp 2411–2420

  • Gupta H, Kottwani A, Gogia S, Chaudhari S (2016) Text analysis and information retrieval of text data. In: 2016 international conference on wireless communications, signal processing and networking (WiSPNET). IEEE, pp 788–792

  • Hajič J, Ciaramita M, Johansson R, Kawahara D, Martí MA, Màrquez L, Meyers A, Nivre J, Padó S, Štěpánek J, Straňák P, Surdeanu M, Xue N, Zhang Y (2009) The CoNLL-2009 shared task: syntactic and semantic dependencies in multiple languages. In: Proceedings of the thirteenth conference on computational natural language learning: shared task, CoNLL ’09, USA. Association for Computational Linguistics, pp 1–18

  • Halácsy P, Trón V (2006) Benefits of deep NLP-based lemmatization for information retrieval. Citeseer, In CLEF (Working Notes)

    Google Scholar 

  • Han B, Baldwin T (2011a) Lexical normalisation of short text messages: Makn sens a# twitter. In: Proceedings of the 49th annual meeting of the Association for Computational Linguistics: human language technologies, pp 368–378

  • Han B, Baldwin T (2011b) Lexical normalisation of short text messages: Makn sens a #twitter. In: Proceedings of the 49th annual meeting of the Association for Computational Linguistics: human language technologies, Portland, Oregon, USA. Association for Computational Linguistics, pp 368–378

  • Hart PE, Nilsson NJ, Raphael B (1968) A formal basis for the heuristic determination of minimum cost paths. IEEE Trans Syst Sci Cybern 4(2):100–107

    Google Scholar 

  • Helgadóttir S (2012) Icelandic frequency dictionary 2012.11-training/testing sets

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Google Scholar 

  • How Y, yen Kan M (2005) Optimizing predictive text entry for short message service on mobile phones. In: In human computer interfaces international (HCII 05). 2005: Las Vegas

  • Huang Z, Xu W, Yu K (2015) Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991

  • Ide N, Véronis J (1994) Multext: multilingual text tools and corpora. In: COLING 1994 vol 1: the 15th international conference on computational linguistics

  • Jahjah V, Khoury R, Lamontagne L (2016) Word normalization using phonetic signatures. In: Canadian conference on artificial intelligence. Springer, pp 180–185

  • Jiampojamarn S, Kondrak G, Sherif T (2007) Applying many-to-many alignments and hidden markov models to letter-to-phoneme conversion. In: Human language technologies 2007: the conference of the North American chapter of the Association for Computational Linguistics; proceedings of the main conference, pp 372–379

  • Jiampojamarn S, Cherry C, Kondrak G (2010) Integrating joint n-gram features into a discriminative training framework. Human language technologies: the 2010 annual conference of the North American chapter of the Association for Computational Linguistics. Los Angeles, California. Association for Computational Linguistics, pp 697–700

    Google Scholar 

  • Jing H, Lopresti D, Shih C (2003) Summarization of noisy documents: a pilot study. In: Proceedings of the HLT-NAACL 03 text summarization workshop, pp 25–32

  • Joachims T (2002) Optimizing search engines using clickthrough data. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining, pp 133–142

  • Johnson R, Zhang T (2005) A high-performance semi-supervised learning method for text chunking. In: Proceedings of the 43rd annual meeting of the Association for Computational Linguistics (ACL’05), pp 1–9

  • Johnson AE, Pollard TJ, Shen L, Li-Wei HL, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG (2016) Mimic-iii, a freely accessible critical care database. Sci Data 3(1):1–9

    Google Scholar 

  • Jones KS (1972) A statistical interpretation of term specificity and its application in retrieval. J Doc 28:11–21

    Google Scholar 

  • Jongejan B, Dalianis H (2009) Automatic training of lemmatization rules that handle morphological changes in pre-, in-and suffixes alike. In: Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP, pp 145–153

  • Jose G, Raj NS (2014) Lexico-syntactic normalization model for noisy SMS text. In: 2014 international conference on electronics, communication and computational engineering (ICECCE). IEEE, pp 163–168

  • Judge J, Cahill A, van Genabith J (2006) QuestionBank: creating a corpus of parse-annotated questions. In: Proceedings of the 21st international conference on computational linguistics and 44th annual meeting of the Association for Computational Linguistics, Sydney, Australia. Association for Computational Linguistics, pp 497–504

  • Juršič M, Mozetič I, Lavrač N (2007) Learning ripple down rules for efficient lemmatization. In: Proceedings of the 10th international multiconference information society, IS, pp 206–209

  • Kanakaraddi SG, Nandyal SS (2018) Survey on parts of speech tagger techniques. In: 2018 international conference on current trends towards converging technologies (ICCTCT). IEEE, pp 1–6

  • Kanis J, Müller L (2005) Automatic lemmatizer construction with focus on OOV words lemmatization. In: International Conference on Text, Speech and Dialogue, pp 132–139. Springer

  • Kanis J, Skorkovská L (2010) Comparison of different lemmatization approaches through the means of information retrieval performance. In: Sojka P, Horák A, Kopeček I, Pala K (eds) Text, Speech and Dialogue, Berlin, Heidelberg. Springer, Berlin Heidelberg, pp 93–100

    Google Scholar 

  • Kaufmann M, Kalita J (2010) Syntactic normalization of twitter messages. In: International conference on natural language processing, Kharagpur, India, Volume, p 16

  • Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, Ye Q, Liu TY (2017) Lightgbm: A highly efficient gradient boosting decision tree. Adv Neural Inf Process Syst 30:3146–3154

    Google Scholar 

  • Kestemont M, De Pauw G, van Nie R, Daelemans W (2017) Lemmatization for variation-rich languages using deep learning. Digital Scholarship in the Humanities 32(4):797–815

    Google Scholar 

  • Khapra M, Kulkarni A, Sohoney S, Bhattacharyya P (2010) All words domain adapted WSD: Finding a middle ground between supervision and unsupervision. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, pp 1532–1541. Association for Computational Linguistics

  • Khoury R (2015) Phonetic normalization of microtext. In: Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015, pp 1600–1601

  • Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp 1746–1751. Association for Computational Linguistics

  • Kim JD, Ohta T, Tateisi Y, Tsujii J (2003) Genia corpus—a semantically annotated corpus for bio-textmining. Bioinformatics 19(suppl-1):i180–i182

  • Kim Y, Jernite Y, Sontag D, Rush AM (2016) Character-aware neural language models. In: Thirtieth AAAI conference on artificial intelligence

  • Kirov C, Cotterell R, Sylak-Glassman J, Walther G, Vylomova E, Xia P, Faruqui M, Mielke SJ, McCarthy AD, Kübler S, et al. (2018) Unimorph 2.0: universal morphology. arXiv preprint arXiv:1810.11101

  • Kiss T, Strunk J (2002) Viewing sentence boundary detection as collocation identification. In: Proceedings of KONVENS, vol 2002. Citeseer, pp 75–82

  • Kiss T, Strunk J (2006) 12. Unsupervised multilingual sentence boundary detection. Comput Linguist 32(4):485–525. https://doi.org/10.1162/coli.2006.32.4.485. https://direct.mit.edu/coli/article-pdf/32/4/485/1798345/coli.2006.32.4.485.pdf

  • Klambauer G, Unterthiner T, Mayr A, Hochreiter S (2017) Self-normalizing neural networks. In: Proceedings of the 31st international conference on neural information processing systems, pp 972–981

  • Knoll BC, Lindemann EA, Albert AL, Melton GB, Pakhomov SVS (2019) Recurrent deep network models for clinical nlp tasks: Use case with sentence boundary disambiguation. Studies in health technology and informatics 264(31437913):198–202. https://doi.org/10.3233/SHTI190211

    Article  Google Scholar 

  • Koehn P, Hoang H, Birch A, Callison-Burch C, Federico M, Bertoldi N, Cowan B, Shen W, Moran C, Zens R et al (2007) Moses: Open source toolkit for statistical machine translation. In: Proceedings of the 45th annual meeting of the Association for Computational Linguistics companion volume proceedings of the demo and poster sessions, pp 177–180

  • Koeling R (2000) Chunking with maximum entropy models. In: Fourth conference on computational natural language learning and the second learning language in logic workshop

  • Kondratyuk D (2019) Cross-lingual lemmatization and morphology tagging with two-stage multilingual BERT fine-tuning. In: Proceedings of the 16th workshop on computational research in phonetics, phonology, and morphology, pp 12–18

  • Kondratyuk D, Straka M (2019) 75 languages, 1 model: parsing universal dependencies universally. arXiv preprint arXiv:1904.02099

  • Kondratyuk D, Gavenčiak T, Straka M, Hajič J (2018) Lemmatag: jointly tagging and lemmatizing for morphologically-rich languages with BRNNs. arXiv preprint arXiv:1808.03703

  • Krallinger M, Rabal O, Lourenco A, Oyarzabal J, Valencia A (2017) Information retrieval and text mining technologies for chemistry. Chem Rev 117(12):7673–7761

    Google Scholar 

  • Kudo T, Matsumoto Y (2000) Use of support vector learning for chunk identification. In: Fourth conference on computational natural language learning and the second learning language in logic workshop

  • Kudo T, Matsumoto Y (2001) Chunking with support vector machines. In: Second meeting of the North American chapter of the Association for Computational Linguistics

  • Kupiec J (1992) Robust part-of-speech tagging using a hidden markov model. Comput Speech Lang 6(3):225–242. https://doi.org/10.1016/0885-2308(92)90019-Z

    Article  Google Scholar 

  • Kuru O, Can OA, Yuret D (2016) CharNER: character-level named entity recognition. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: Technical Papers, Osaka, Japan. The COLING 2016 Organizing Committee, pp 911–921

  • Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning, ICML ’01, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc, pp 282–289

  • LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Google Scholar 

  • Lee YS, Wu YC (2007) A robust multilingual portable phrase chunking system. Expert Syst Appl 33(3):590–599

    Google Scholar 

  • Leeman-Munk S, Lester J, Cox J (2015) Ncsu_sas_sam: Deep encoding and reconstruction for normalization of noisy text. In: Proceedings of the workshop on noisy user-generated text, pp 154–161

  • Li C, Liu Y (2012) Normalization of text messages using character-and phone-based machine translation approaches. In: Thirteenth annual conference of the international speech communication association

  • Lin JCW, Shao Y, Zhang J, Yun U (2020) Enhanced sequence labeling based on latent variable conditional random fields. Neurocomputing 403:431–440

    Google Scholar 

  • Ling W, Dyer C, Black AW, Trancoso I (2015a) Two/too simple adaptations of word2vec for syntax problems. In: Proceedings of the 2015 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, pp 1299–1304

  • Ling W, Luís T, Marujo L, Astudillo RF, Amir S, Dyer C, Black AW, Trancoso I (2015b) Finding function in form: compositional character models for open vocabulary word representation. arXiv preprint arXiv:1508.02096

  • Littlestone N (1988) Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm. Mach Learn 2(4):285–318

    Google Scholar 

  • Liu Y, Stolcke A, Shriberg E, Harper M (2004) Comparing and combining generative and posterior probability models: some advances in sentence boundary detection in speech. In: Proceedings of the 2004 conference on empirical methods in natural language processing, pp 64–71

  • Liu Y, Stolcke A, Shriberg E, Harper M (2005) Using conditional random fields for sentence boundary detection in speech. In: Proceedings of the 43rd annual meeting of the Association for Computational Linguistics (ACL’05), pp 451–458

  • Liu Y, Chawla NV, Harper MP, Shriberg E, Stolcke A (2006) A study in machine learning from imbalanced data for sentence boundary detection in speech. Comput Speech Lang 20(4):468–494. https://doi.org/10.1016/j.csl.2005.06.002

    Article  Google Scholar 

  • Liu F, Weng F, Wang B, Liu Y (2011) Insertion, deletion, or substitution? normalizing text messages without pre-categorization nor supervision. In: Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, USA. Association for Computational Linguistics, pp 71–76

  • Liu F, Weng F, Jiang X (2012) A broad-coverage normalization system for social media language. In: Proceedings of the 50th annual meeting of the association for computational linguistics (vol 1: Long Papers), pp 1035–1044

  • Liu H, Dacon J, Fan W, Liu H, Liu Z, Tang J (2019) Does gender matter? towards fairness in dialogue systems. arXiv preprint arXiv:1910.10486

  • Liu Y, Li G, Zhang X (2020) Semi-Markov CRF model based on stacked neural Bi-LSTM for sequence labeling. In: 2020 IEEE 3rd international conference of safe production and informatization (IICSPI), pp 19–23

  • Lourentzou I, Manghnani K, Zhai C (2019) Adapting sequence to sequence models for text normalization in social media. Proceedings of the international AAAI conference on web and social media, vol 13, pp 335–345

    Google Scholar 

  • Luhn HP (1957) A statistical approach to mechanized encoding and searching of literary information. IBM J Res Dev 1(4):309–317

    MathSciNet  Google Scholar 

  • Luong MT, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025

  • Lusetti M, Ruzsics T, Göhring A, Samardzic T, Stark E (2018) Encoder-decoder methods for text normalization. In: VarDial@COLING 2018

  • Lyras DP, Sgarbas KN, Fakotakis ND (2007) Using the Levenshtein edit distance for automatic lemmatization: a case study for modern Greek and English. In: 19th IEEE international conference on tools with artificial intelligence (ICTAI 2007), vol 2. IEEE, pp 428–435

  • Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. arXiv preprint arXiv:1603.01354

  • Ma J, Zhu J, Xiao T, Yang N (2013) Easy-first POS tagging and dependency parsing with beam search. In: Proceedings of the 51st annual meeting of the Association for Computational Linguistics (vol 2: Short Papers), pp 110–114

  • Maamouri M, Bies A, Buckwalter T, Mekki W (2004) The Penn Arabic Treebank: Building a large-scale annotated Arabic corpus. In: NEMLAR conference on Arabic language resources and tools, vol 27. Cairo, pp 466–467

  • Maamouri M, Krouna S, Tabessi D, Hamrouni N, Habash N (2012) Egyptian Arabic morphological annotation guidelines

  • Mahmood A, Khan HU, ur Rehman Z, Khan W (2017) Query based information retrieval and knowledge extraction using hadith datasets. In: 2017 13th international conference on emerging technologies (ICET), pp 1–6

  • Malaviya C, Wu S, Cotterell R (2019) A simple joint model for improved contextual neural lemmatization. arXiv preprint arXiv:1904.02306

  • Manandhar S, Džeroski S, Erjavec T (1998) Learning multilingual morphology with clog. In: International conference on inductive logic programming. Springer, pp 135–144

  • Manjavacas E, Kádár Á, Kestemont M (2019) Improving lemmatization of non-standard languages with joint learning. arXiv preprint arXiv:1903.06939

  • Manning CD (2011) Part-of-speech tagging from 97% to 100%: Is it time for some linguistics? In: Gelbukh AF (ed) Computational linguistics and intelligent text processing. Springer, Berlin, pp 171–189

    Google Scholar 

  • Mao R, Li X (2021) Bridging towers of multi-task learning with a gating mechanism for aspect-based sentiment analysis and sequential metaphor identification. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 13534–13542

    Google Scholar 

  • Mao R, Lin C, Guerin F (2018) Word embedding and WordNet based metaphor identification and interpretation. In: Proceedings of the 56th annual meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 1222–1231

  • Mao R, Lin C, Guerin F (2019) End-to-end sequential metaphor identification inspired by linguistic theories. In: Proceedings of the 57th annual meeting of the Association for Computational Linguistics (Long Papers), pp 3888–3898

  • Mao R, Lin C, Guerin F (2021) Combining pre-trained word embeddings and linguistic features for sequential metaphor identification. arXiv preprint arXiv:2104.03285

  • Mao R, Li X, Ge M, Cambria E (2022) Metapro: A computational metaphor processing model for text pre-processing. Inf Fus 86–87:30–43. https://doi.org/10.1016/j.inffus.2022.06.002

    Article  Google Scholar 

  • Marcus MP, Santorini B, Marcinkiewicz MA (1993) Building a large annotated corpus of English: The Penn Treebank. Comput Linguist 19(2):313–330

    Google Scholar 

  • Martı MA, Taulé M, Márquez L, Bertran M (2007) CESS-ECE: a multilingual and multilevel annotated corpus. Available for download from: http://www.lsi.upc.edu/mbertran/cess-ece

  • Matsoukas S, Bulyko I, Xiang B, Nguyen K, Schwartz R, Makhoul J (2007) Integrating speech recognition and machine translation. In: 2007 IEEE international conference on acoustics, speech and signal processing-ICASSP’07, vol 4. IEEE, pp IV–1281

  • McCallum A, Freitag D, Pereira FC (2000) Maximum entropy markov models for information extraction and segmentation. Icml 17:591–598

    Google Scholar 

  • McCarthy AD, Vylomova E, Wu S, Malaviya C, Wolf-Sonkin L, Nicolai G, Kirov C, Silfverberg M, Mielke SJ, Heinz J, et al. (2019) The SIGMORPHON 2019 shared task: Morphological analysis in context and cross-lingual transfer for inflection. arXiv preprint arXiv:1910.11493

  • McCord MC (1990) Slot grammar, natural language and logic. Springer, pp 118–145

  • McDonald R, Crammer K, Pereira F (2005) Flexible text segmentation with structured multilabel classification. In: Proceedings of human language technology conference and conference on empirical methods in natural language processing, pp 987–994

  • Meena R, Skantze G, Gustafson J (2014) Data-driven models for timing feedback responses in a map task dialogue system. Comput Speech Lang 28(4):903–922

    Google Scholar 

  • Meftah S, Semmar N (2018) A neural network model for part-of-speech tagging of social media texts. In: Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA)

  • Mhatre M, Phondekar D, Kadam P, Chawathe A, Ghag K (2017) Dimensionality reduction for sentiment analysis using pre-processing techniques. In: 2017 international conference on computing methodologies and communication (ICCMC). IEEE, pp 16–21

  • Mikheev A (2000) Tagging sentence boundaries. In: 1st meeting of the North American chapter of the Association for Computational Linguistics

  • Mikheev A (2002) Periods, capitalized words, etc. Comput Linguist 28(3):289–318

    Google Scholar 

  • Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. In: Proceedings of workshop at ICLR 2013

  • Milintsevich K, Sirts K (2021) Enhancing sequence-to-sequence neural lemmatization with external resources. arXiv preprint arXiv:2101.12056

  • Mittal A, Bhatt P, Kumar P (2014) Phonetic matching and syntactic tree similarity based QA system for SMS queries. In: 2014 international conference on green computing communication and electrical engineering (ICGCCEE). IEEE, pp 1–6

  • Mladenic D (2002) Automatic word lemmatization. In: Proceedings of the 5th international multi-conference information society, IS-2002 B, pp 153–159

  • Molina A, Pla F (2002) Shallow parsing using specialized HMMs. J Mach Learn Res 2(Mar):595–613

  • Morency LP, Quattoni A, Darrell T (2007) Latent-dynamic discriminative models for continuous gesture recognition. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8

  • Mubarok MS, Adiwijaya, Aldhi MD (2017) Aspect-based sentiment analysis to review products using Naïve Bayes. AIP Conf Proc 1867:020060

  • Muis AO, Lu W (2016) Weak semi-Markov CRFs for noun phrase chunking in informal text. In: Proceedings of the 2016 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, pp 714–719

  • Müller T, Schmid H, Schütze H (2013) Efficient higher-order crfs for morphological tagging. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 322–332

  • Müller T, Cotterell R, Fraser A, Schütze H (2015) Joint lemmatization and morphological tagging with Lemming. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 2268–2274

  • Nakagawa T, Kudo T, Matsumoto Y (2001) Unknown word guessing and part-of-speech tagging using support vector machines. In: NLPRS. Citeseer, pp 325–331

  • Nandhini BS, Sheeba J (2015) Cyberbullying detection and classification using information retrieval algorithm. In: Proceedings of the 2015 international conference on advanced research in computer science engineering & technology (ICARCSET 2015), pp 1–5

  • Ngai G, Florian R (2001) Transformation based learning in the fast lane. In: Second meeting of the North American chapter of the Association for Computational Linguistics

  • Nguyen DQ, Vu T, Nguyen DQ, Dras M, Johnson M (2017) From word segmentation to POS tagging for Vietnamese. arXiv preprint arXiv:1711.04951

  • Nicolai G, Kondrak G (2016) Leveraging inflection tables for stemming and lemmatization. In: Proceedings of the 54th annual meeting of the Association for Computational Linguistics (vol 1: Long Papers), pp 1138–1147

  • Niehues J, Cho E (2017) Exploiting linguistic resources for neural machine translation using multi-task learning. arXiv preprint arXiv:1708.00993

  • Nivre J, Hall J, Kübler S, McDonald R, Nilsson J, Riedel S, Yuret D (2007) The CoNLL 2007 shared task on dependency parsing. In: Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), Prague, Czech Republic. Association for Computational Linguistics, pp 915–932

  • Nivre J, De Marneffe MC, Ginter F, Goldberg Y, Hajic J, Manning CD, McDonald R, Petrov S, Pyysalo S, Silveira N, et al. (2016) Universal dependencies v1: a multilingual treebank collection. In: Proceedings of the tenth international conference on language resources and evaluation (LREC’16), pp 1659–1666

  • Nocedal J, Wright S (2006) Numerical optimization. Springer, New York

    MATH  Google Scholar 

  • O’Connor B, Krieger M, Ahn D (2010) Tweetmotif: exploratory search and topic summarization for twitter. In: Fourth international AAAI conference on weblogs and social media

  • Otter DW, Medina JR, Kalita JK (2020) A survey of the usages of deep learning for natural language processing. IEEE transactions on neural networks and learning systems 32(2):604–624

    MathSciNet  Google Scholar 

  • Owoputi O, O’Connor B, Dyer C, Gimpel K, Schneider N, Smith NA (2013) Improved part-of-speech tagging for online conversational text with word clusters. In: Proceedings of the 2013 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, pp 380–390

  • Palmer DD, Hearst MA (1994) Adaptive sentence boundary disambiguation. arXiv preprint cmp-lg/9411022

  • Palmer DD, Hearst MA (1997) Adaptive multilingual sentence boundary disambiguation. Comput. Linguist. 23(2):241–267

    Google Scholar 

  • Papineni K, Roukos S, Ward T, Zhu WJ (2002) Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp 311–318

  • Pennell DL, Liu Y (2010) Normalization of text messages for text-to-speech. In: 2010 IEEE international conference on acoustics, speech and signal processing. IEEE, pp 4842–4845

  • Pennell D, Liu Y (2011) A character-level machine translation approach for normalization of SMS abbreviations. In: Proceedings of 5th international joint conference on natural language processing, pp 974–982

  • Pennell DL, Liu Y (2014) Normalization of informal text. Comput Speech Lang 28(1):256–277

    Google Scholar 

  • Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  • Peters R, Nagel N (2014) Das digitale, referenzkorpus mittelniederdeutsch/niederrheinisch (ReN)’. Jahrbuch für Germanistische Sprachgeschichte 5(1):165–175

    Google Scholar 

  • Petrov S, Das D, McDonald R (2011) A universal part-of-speech tagset. arXiv preprint arXiv:1104.2086

  • Petrović S, Osborne M, Lavrenko V (2010) The Edinburgh twitter corpus. In: Proceedings of the NAACL HLT 2010 workshop on computational linguistics in a world of social media, pp 25–26

  • Philips L (1990) Hanging on the metaphone. Comput Lang 7(12):39–43

    Google Scholar 

  • Philips L (2000) The double metaphone search algorithm. C/C++ users J 18(6):38–43

  • Platt J (1999) Fast training of support vector machines using sequential minimal optimization. Adv Kernel Methods 185–208

  • Plisson J, Lavrac N, Mladenic D et al (2004) A rule based approach to word lemmatization. Proceedings of IS 3:83–86

    Google Scholar 

  • Priyanshu A, Das VR, Rajiv Moghe S, Rathod H, Medicherla SS, Shail Chhabra M, Shastri S (2020) Stance classification with improved elementary classifiers using lemmatization (grand challenge). In: 2020 IEEE sixth international conference on multimedia big data (BigMM), pp 466–470

  • Punyakanok V, Roth D (2000) The use of classifiers in sequential inference. Adv Neural Inf Process Syst 13:995–1001

    Google Scholar 

  • Pütz T, De Kok D, Pütz S, Hinrichs E (2018) Seq2seq or perceptrons for robust lemmatization. an empirical examination. In: Proceedings of the 17th international workshop on treebanks and linguistic theories (TLT 2018), pp 193–207

  • Qi P, Dozat T, Zhang Y, Manning CD (2018) Universal Dependency parsing from scratch. In: Proceedings of the CoNLL 2018 shared task: multilingual parsing from raw text to universal dependencies, Brussels, Belgium. Association for Computational Linguistics, pp 160–170

  • Qi P, Zhang Y, Zhang Y, Bolton J, Manning CD (2020) Stanza: a python natural language processing toolkit for many human languages. arXiv preprint arXiv:2003.07082

  • Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Google Scholar 

  • Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training. Report, OpenAI

    Google Scholar 

  • Raffel C, Luong MT, Liu PJ, Weiss RJ, Eck D (2017) Online and linear-time attention by enforcing monotonic alignments. In: International conference on machine learning, pp 2837–2846. PMLR

  • Ramshaw LA, Marcus MP (1999) Text chunking using transformation-based learning, Natural language processing using very large corpora, 157–176. Springer

    Google Scholar 

  • Rastogi P, Cotterell R, Eisner J (2016) Weighting finite-state transductions with neural context. In: Proceedings of the 2016 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, pp 623–633

  • Ratinov L, Roth D (2009) Design challenges and misconceptions in named entity recognition. In: Proceedings of the thirteenth conference on computational natural language learning (CoNLL-2009), pp 147–155

  • Ratnaparkhi A (1996) A maximum entropy model for part-of-speech tagging. In: Conference on empirical methods in natural language processing

  • Read J, Dridan R, Oepen S, Solberg LJ (2012) Sentence boundary detection: a long solved problem? In Proceedings of COLING 2012: posters, pp 985–994

  • Rei M (2017) Semi-supervised multitask learning for sequence labeling. arXiv preprint arXiv:1704.07156

  • Reynar JC, Ratnaparkhi A (1997) A maximum entropy approach to identifying sentence boundaries. arXiv preprint cmp-lg/9704002

  • Riley M (1989) Some applications of tree-based modelling to speech and language. In: Speech and natural language: proceedings of a workshop held at Cape Cod, Massachusetts, October 15–18

  • Ritter A, Clark S, Etzioni O, et al. (2011) Named entity recognition in tweets: an experimental study. In: Proceedings of the 2011 conference on empirical methods in natural language processing, pp 1524–1534

  • Roberts L (2016) Syntactic processing. Cambridge handbooks in language and linguistics. Cambridge University Press, Cambridge, pp 227–247

    Google Scholar 

  • Rosa KD, Ellen J (2009) Text classification methodologies applied to micro-text in military chat. In: 2009 international conference on machine learning and applications, pp 710–714

  • Rosa R, Žabokrtskỳ Z (2019) Unsupervised lemmatization as embeddings-based word clustering. arXiv preprint arXiv:1908.08528

  • Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386

    Google Scholar 

  • Rudrapal D, Jamatia A, Chakma K, Das A, Gambäck B (2015) Sentence boundary detection for social media text. In: Proceedings of the 12th international conference on natural language processing, Trivandrum, India. NLP Association of India, pp 254–260

  • Rush AM, Reichart R, Collins M, Globerson A (2012) Improved parsing and POS tagging using inter-sentence consistency constraints. In: Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning, pp 1434–1444

  • Ruzsics T, Samardzic T (2017) Neural sequence-to-sequence learning of internal word structure. In: Proceedings of the 21st conference on computational natural language learning (CoNLL 2017), pp 184–194

  • Sadvilkar N, Neumann M (2020) PySBD: pragmatic sentence boundary disambiguation. arXiv preprint arXiv:2010.09657

  • Salzberg SL (1994) C4.5: Programs for machine learning by J. Ross Quinlan. Morgan Kaufmann Publishers, inc., 1993. Mach Learn 16(3): 235–240. https://doi.org/10.1007/BF00993309

  • Sanchez G (2019) Sentence boundary detection in legal text. Proceedings of the natural legal language processing workshop 2019:31–38

    Google Scholar 

  • Sang ETK (2000) Text chunking by system combination. In: Fourth conference on computational natural language learning and the second learning language in logic workshop

  • Sang EF, Buchholz S (2000) Introduction to the CoNLL-2000 shared task: Chunking. arXiv preprint cs/0009008

  • Sang EF, Veenstra J (1999) Representing text chunks. arXiv preprint cs/9907006

  • Santos CNd, Guimaraes V (2015) Boosting named entity recognition with neural character embeddings. arXiv preprint arXiv:1505.05008

  • Sarawagi S, Cohen WW (2004) Semi-Markov conditional random fields for information extraction. Adv Neural Inf Process Syst 17:1185–1192

    Google Scholar 

  • Satapathy R, Guerreiro C, Chaturvedi I, Cambria E (2017) Phonetic-based microtext normalization for twitter sentiment analysis. In: 2017 IEEE international conference on data mining workshops (ICDMW), pp 407–413

  • Satapathy R, Li Y, Cavallari S, Cambria E (2019a) Seq2seq deep learning models for microtext normalization. In: 2019 international joint conference on neural networks (IJCNN), pp 1–8. IEEE

  • Satapathy R, Singh A, Cambria E (2019b) Phonsenticnet: a cognitive approach to microtext normalization for concept-level sentiment analysis. In: International conference on computational data and social networks, pp 177–188. Springer

  • Satapathy R, Cambria E, Nanetti A, Hussain A (2020) A review of shorthand systems: from brachygraphy to microtext and beyond. Cogn Comput 12(4):778–792

    Google Scholar 

  • Savary A, Zaborowski B, Krawczyk-Wieczorek A, Makowiecki F (2012) Sejfek - a lexicon and a shallow grammar of polish economic multi-word units. In: Proceedings of the 3rd workshop on cognitive aspects of the lexicon, pp 195–214

  • Schapire RE, Singer Y (1999) Improved boosting algorithms using confidence-rated predictions. Mach Learn 37(3):297–336

    MATH  Google Scholar 

  • Schmid H (2000) Unsupervised learning of period disambiguation for tokenisation. Internal Report, IMS-CL

    Google Scholar 

  • Schmitt M, Constant M (2019) Neural lemmatization of multiword expressions. In: Proceedings of the joint workshop on multiword expressions and wordnet (MWE-WN 2019), pp 142–148

  • Seddah D, Tsarfaty R, Kübler S, Candito M, Choi J, Farkas R, Foster J, Goenaga I, Gojenola K, Goldberg Y et al. (2013) Overview of the SPMRL 2013 shared task: cross-framework evaluation of parsing morphologically rich languages. In: Proceedings of the fourth workshop on statistical parsing of morphologically-rich languages. Association for Computational Linguistics

  • Sennrich R, Firat O, Cho K, Birch A, Haddow B, Hitschler J, Junczys-Dowmunt M, Läubli S, Barone AVM, Mokry J et al. (2017) Nematus: a toolkit for neural machine translation. arXiv preprint arXiv:1703.04357

  • Sha F, Pereira F (2003) Shallow parsing with conditional random fields. In: Proceedings of the 2003 human language technology conference of the North American Chapter of the Association for Computational Linguistics, pp 213–220

  • Shao Y, Hardmeier C, Tiedemann J, Nivre J (2017) Character-based joint segmentation and pos tagging for chinese using bidirectional rnn-crf. arXiv preprint arXiv:1704.01314

  • Shen L, Satta G, Joshi A (2007) Guided learning for bidirectional sequence classification. In: Proceedings of the 45th annual meeting of the Association of Computational Linguistics, Prague, Czech Republic. Association for Computational Linguistics, pp 760–767

  • Shewchuk JR et al. (1994) An introduction to the conjugate gradient method without the agonizing pain

  • Silveira N, Dozat T, De Marneffe MC, Bowman SR, Connor M, Bauer J, Manning CD (2014) A gold standard dependency corpus for English. In: LREC, pp 2897–2904. Citeseer

  • Skut W, Krenn B, Brants T, Uszkoreit H (2002) 05. An annotation scheme for free word order languages. Proceedings of the 5th conference on applied natural language processing. https://doi.org/10.3115/974557.974571

  • Søgaard A (2010) Simple semi-supervised training of part-of-speech taggers. In: Proceedings of the ACL 2010 conference short papers, Uppsala, Sweden. Association for Computational Linguistics, pp 205–208

  • Song M, Song IY, Hu X, Allen RB (2005) Integrating text chunking with mixture hidden Markov models for effective biomedical information extraction. In: International conference on computational science. Springer, pp 976–984

  • Spoustová Dj, Hajič J, Raab J, Spousta M (2009) Semi-supervised training for the averaged perceptron POS tagger. In: Proceedings of the 12th conference of the European chapter of the ACL (EACL 2009), Athens, Greece. Association for Computational Linguistics, pp 763–771

  • Stamatatos E, Fakotakis N, Kokkinakis G (1999) Automatic extraction of rules for sentence boundary disambiguation. In: Proceedings of the workshop on machine learning in human language technology. Citeseer, pp 88–92

  • Stevenson M, Gaizauskas R (2000) Experiments on sentence boundary detection. In: Sixth applied natural language processing conference, pp 84–89

  • Strassel S (2003) Simple metadata annotation specification version 5.0–may 14, 2003

  • Subramanya A, Petrov S, Pereira F (2010) Efficient graph-based semi-supervised learning of structured tagging models. In: Proceedings of the 2010 conference on empirical methods in natural language processing, EMNLP ’10, USA. Association for Computational Linguistics, pp 167–176

  • Sun X, Morency LP, Okanohara D, Tsuruoka Y, Tsujii J (2008) Modeling latent-dynamic in shallow parsing: a latent conditional model with improved inference. In: Proceedings of the 22nd international conference on computational linguistics (Coling 2008), pp 841–848

  • Sun X, Sun S, Yin M, Yang H (2020) Hybrid neural conditional random fields for multi-view sequence labeling. Knowl-Based Syst 189:105151

    Google Scholar 

  • Sutton C, McCallum A, Rohanimanesh K (2007) Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data. J Mach Learn Res 8(3)

  • Suzuki J, Isozaki H (2008) Semi-supervised sequential labeling and segmentation using giga-word scale unlabeled data. In: Proceedings of ACL-08: HLT, pp 665–673

  • Syed AZ, Aslam M, Martinez-Enriquez AM (2014) Associating targets with sentiunits: a step forward in sentiment analysis of urdu text. Artif Intell Rev 41(4):535–561

    Google Scholar 

  • Symeonidis S, Effrosynidis D, Arampatzis A (2018) A comparative evaluation of pre-processing techniques and their interactions for twitter sentiment analysis. Expert Syst Appl 110:298–310

    Google Scholar 

  • Taghipour K, Ng HT (2015) Semi-supervised word sense disambiguation using word embeddings in general and specific domains. In: Proceedings of the 2015 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, pp 314–323

  • Taylor P, Black AW, Caley R (1998) The architecture of the festival speech synthesis system. In: The third ESCA/COCOSDA workshop (ETRW) on speech synthesis

  • Telljohann H, Hinrichs E, Kübler S, Kübler R (2004) The tüba-d/z treebank: annotating german with a context-free backbone. In: Proceedings of the fourth international conference on language resources and evaluation (LREC 2004). Citeseer

  • Toutanova K, Cherry C (2009) A global model for joint lemmatization and part-of-speech prediction. In: Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP, pp 486–494

  • Toutanova K, Johnson M (2007) A Bayesian LDA-based model for semi-supervised part-of-speech tagging. Adv Neural Inf Process Syst 20:1521–1528

    Google Scholar 

  • Toutanvoa K, Manning CD (2000) Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. 2000 Joint SIGDAT conference on empirical methods in natural language processing and very large corpora. China. Association for Computational Linguistics, Hong Kong, pp 63–70

    Google Scholar 

  • Toutanova K, Klein D, Manning CD, Singer Y (2003) Feature-rich part-of-speech tagging with a cyclic dependency network. In: Proceedings of the 2003 conference of the North American chapter of the Association for Computational Linguistics on human language technology - vol 1, NAACL ’03, USA. Association for Computational Linguistics, pp 173–180

  • Treviso MV, Shulby C, Aluísio SM (2016) Sentence segmentation in narrative transcripts from neuropsychological tests using recurrent convolutional neural networks. arXiv preprint arXiv:1610.00211

  • Treviso MV, Shulby CD, Aluisio SM (2017) Evaluating word embeddings for sentence boundary detection in speech transcripts

  • Tsochantaridis I, Hofmann T, Joachims T, Altun Y (2004) Support vector machine learning for interdependent and structured output spaces. In: Proceedings of the twenty-first international conference on Machine learning, pp 104

  • Tsuruoka Y, Tsujii J (2005) Bidirectional inference with the easiest-first strategy for tagging sequence data. In: Proceedings of the conference on human language technology and empirical methods in natural language processing, HLT ’05, USA, pp 467–474. Association for Computational Linguistics

  • Ueberwasser S, Stark E (2017) What’s up, Switzerland? A corpus-based research project in a multilingual country. Linguistik 84(5):105

    Google Scholar 

  • Utgoff PE, Berkman NC, Clouse JA (1997) Decision tree induction based on efficient tree restructuring. Mach Learn 29(1):5–44. https://doi.org/10.1023/A:1007413323501

    Article  MATH  Google Scholar 

  • van Halteren H (2000) A default first order family weight determination procedure for WPDV models. In: Fourth conference on computational natural language learning and the second learning language in logic workshop

  • van Halteren H, Daelemans W, Zavrel J (2001) June) Improving accuracy in word class tagging through the combination of machine learning systems. Comput. Linguist. 27(2):199–229. https://doi.org/10.1162/089120101750300508

    Article  Google Scholar 

  • Van Halteren H (2000) Chunking with WPDV models. In: Fourth conference on computational natural language learning and the second learning language in logic workshop

  • Van Kerckvoorde CM (2019) An introduction to middle Dutch. De Gruyter Mouton

  • Vinyals O, Fortunato M, Jaitly N (2015) Pointer networks. In: Advances in neural information processing systems, pp 2692–2700

  • Wang P, Ng HT (2013) A beam-search decoder for normalization of social media text with application to machine translation. In: Proceedings of the 2013 conference of the North American chapter of the Association for Computational Linguistics: human language technologies, pp 471–481

  • Wang P, Qian Y, Soong FK, He L, Zhao H (2015) Part-of-speech tagging with bidirectional long short-term memory recurrent neural network. CoRR abs/1510.06168. arXiv:1510.06168

  • Wei W, Wang Z, Mao X, Zhou G, Zhou P, Jiang S (2021) Position-aware self-attention based neural sequence labeling. Pattern Recogn 110:107636. https://doi.org/10.1016/j.patcog.2020.107636

    Article  Google Scholar 

  • Wilcox-O’Hearn A, Hirst G, Budanitsky A (2008) Real-word spelling correction with trigrams: a reconsideration of the mays, damerau, and mercer model. In: International conference on intelligent text processing and computational linguistics. Springer, pp 605–616

  • Winkler W (1990) String comparator metrics and enhanced decision rules in the Fellegi-Sunter model of record linkage. In: Proceedings of the section on survey research methods

  • Wong F, Chao S (2010) isentenizer: an incremental sentence boundary classifier. In: Proceedings of the 6th international conference on natural language processing and knowledge engineering (NLPKE-2010). IEEE, pp 1–7

  • Wong DF, Chao LS, Zeng X (2014) isentenizer-: multilingual sentence boundary detection model. Sci World J 2014:1–10

    Google Scholar 

  • Woolf BP (2009) Chapter 5 - communication knowledge. In: Woolf BP (ed) Building intelligent interactive tutors. Morgan Kaufmann, San Francisco, pp 136–182

    Google Scholar 

  • Wu S, Cotterell R (2019) Exact hard monotonic attention for character-level transduction. arXiv preprint arXiv:1905.06319

  • Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y (2015a) Show, attend and tell: neural image caption generation with visual attention. In: International conference on machine learning. PMLR, pp 2048–2057

  • Xu K, Xia Y, Lee CH (2015b) Tweet normalization with syllables. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th international joint conference on natural language processing (vol 1: Long Papers), pp 920–928

  • Xue Z, Yin D, Davison BD (2011) Normalizing microtext. In: Workshops at the twenty-fifth AAAI conference on artificial intelligence. Citeseer

  • Yang Y, Eisenstein J (2013) A log-linear model for unsupervised text normalization. In: Proceedings of the 2013 conference on empirical methods in natural language processing, pp 61–72

  • Yang Z, Salakhutdinov R, Cohen W (2016) Multi-task cross-lingual sequence tagging from scratch. arXiv preprint arXiv:1603.06270

  • Yang Z, Salakhutdinov R, Cohen WW (2017) Transfer learning for sequence tagging with hierarchical recurrent networks. CoRR abs/1703.06345. arXiv:1703.06345

  • Yang J, Liang S, Zhang Y (2018) Design challenges and misconceptions in neural sequence labeling. arXiv preprint arXiv:1806.04470

  • Yang S, Wang Y, Chu X (2020) A survey of deep learning techniques for neural machine translation. arXiv preprint arXiv:2002.07526

  • Yildiz E, Tantuğ AC (2019) Morpheus: a neural network for jointly learning contextual lemmatization and morphological tagging. In: Proceedings of the 16th workshop on computational research in phonetics, phonology, and morphology, pp 25–34

  • Zalmout N, Habash N (2019) Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging. arXiv preprint arXiv:1910.02267

  • Zalmout N, Habash N (2020) Utilizing subword entities in character-level sequence-to-sequence lemmatization models. In: Proceedings of the 28th international conference on computational linguistics, pp 4676–4682

  • Zeman D, Hajic J, Popel M, Potthast M, Straka M, Ginter F, Nivre J, Petrov S (2018) CoNLL 2018 shared task: multilingual parsing from raw text to universal dependencies. In: Proceedings of the CoNLL 2018 shared task: multilingual parsing from raw text to universal dependencies, pp 1–21

  • Zhai F, Potdar S, Xiang B, Zhou B (2017) Neural models for sequence chunking. arXiv preprint arXiv:1701.04027

  • Zhang T, Damerau F, Johnson DE (2001) Text chunking using regularized winnow. In: Proceedings of the 39th annual meeting of the Association for Computational Linguistics, pp 539–546

  • Zhang T, Damerau F, Johnson D (2002) Text chunking based on a generalization of winnow. J Mach Learn Res 2(Mar): 615–637

  • Zhang C, Baldwin T, Ho H, Kimelfeld B, Li Y (2013) Adaptive parser-centric text normalization. In: Proceedings of the 51st annual meeting of the Association for Computational Linguistics (vol 1: Long Papers), pp 1159–1168

  • Zhao J, Gao Q (2017) Annotation and detection of emotion in text-based dialogue systems with cnn. arXiv preprint arXiv:1710.00987

  • Zhao L, Qiu X, Zhang Q, Huang X (2019) Sequence labeling with deep gated dual path CNN. IEEE/ACM Trans Audio Speech Lang Process 27(12):2326–2335

    Google Scholar 

  • Zhou G, Su J (2000) Error-driven HMM-based chunk tagger with context-dependent lexicon. In: 2000 Joint SIGDAT conference on empirical methods in natural language processing and very large corpora, pp 71–79

  • Zhou N, Wang X, Aw A (2017) Dynamic boundary detection for speech translation. In: 2017 Asia-pacific signal and information processing association annual summit and conference (APSIPA ASC), pp 651–656. IEEE

  • Zhou D, Zhang Z, Zhang ML, He Y (2018) Weakly supervised POS tagging without disambiguation. ACM Trans Asian Low-Resour Lang Inf Process (TALLIP) 17(4):1–19

    Google Scholar 

  • Zhou H, Zhang Y, Li Z, Zhang M (2020) Is POS tagging necessary or even helpful for neural dependency parsing?

Download references

Acknowledgements

This research is supported by the Agency for Science, Technology and Research (A*STAR) under its AME Programmatic Funding Scheme (Project #A18A2b0046).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Nanyang Technological University, Singapore , Singapore

    Xulang Zhang, Rui Mao & Erik Cambria

Authors
  1. Xulang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik Cambria.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, X., Mao, R. & Cambria, E. A survey on syntactic processing techniques. Artif Intell Rev 56, 5645–5728 (2023). https://doi.org/10.1007/s10462-022-10300-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-022-10300-7

Keywords

Search

Navigation