Abstract
Recurrent neural networks (RNN) and one of its variants long short-term memory networks (LSTM) are the main network architectures for NLP due to their strong power to capture the long-ranged contextual dependencies in sequential data. This dissertation focuses on investigat-ing recurrent neural representations for three NLP tasks, namely sentiment analysis, sentence classification and constituency parsing. For sentiment analysis, we propose a context-aware lexicon-based method, which uses an LSTM to learn the strength, intensification and negation of lexicon sentiments in composing the sentiment value of sentences with a simple weighted-sum model. Results show that our model can not only learn such operation details, but also improves significantly over the state-of-the-art LSTM baselines without lexicon features, having achieved the best results on a Twitter benchmark. For syntax-based sentence classification, we propose head-lexicalized bidirectional tree-structured LSTMs, which provide recurrent representations for unlabeled constituent trees. Experiments show that our model learns better representations of tree structures, giving the best results on the Stanford Sentiment Treebank and highly compet-itive results on the TREC question type classification task when the model was published. For constituency parsing, we present two conceptually simple local neural models based on LSTMs, which make local decisions to constituent spans and context free grammar (CFG) rules, respec-tively. Consistent with previous findings, our best model, because of its dense neural input representations that embody global syntactic information, gives highly competitive results.