Abstract:
Neural network based approaches have recently produced record-setting performances in natural language understanding tasks such as word labeling. In the word labeling tas...Show MoreMetadata
Abstract:
Neural network based approaches have recently produced record-setting performances in natural language understanding tasks such as word labeling. In the word labeling task, a tagger is used to assign a label to each word in an input sequence. Specifically, simple recurrent neural networks (RNNs) and convolutional neural networks (CNNs) have shown to significantly outperform the previous state-of-the-art - conditional random fields (CRFs). This paper investigates using long short-term memory (LSTM) neural networks, which contain input, output and forgetting gates and are more advanced than simple RNN, for the word labeling task. To explicitly model output-label dependence, we propose a regression model on top of the LSTM un-normalized scores. We also propose to apply deep LSTM to the task. We investigated the relative importance of each gate in the LSTM by setting other gates to a constant and only learning particular gates. Experiments on the ATIS dataset validated the effectiveness of the proposed models.
Published in: 2014 IEEE Spoken Language Technology Workshop (SLT)
Date of Conference: 07-10 December 2014
Date Added to IEEE Xplore: 02 April 2015
Electronic ISBN:978-1-4799-7129-9