Abstract
We study several deep neural networks approaches to learn neural representations of au-tomatic essay scoring, including convolutional neural network, recurrent convolutional neural network and variational autoencoder based deep neural network models. We learn representations of essays from a hierarchy of two-level representations, i.e. sentence-level to document-level. Our neural models are based on such hierarchies to learn sentence-level representations to document-level representations. We ?rstly build a hierarchical convolutional neural network to learn automatic feature representations of essays following sentence-level to document-level hierarchies. For capturing long dependencies among sentences, we utilize a recurrent neural network to encode document representations and attention mechanism is in-corporated for better differentiating the effect of words and sentences. Finally, we propose a variational autoencoder based deep neural model to learn feature representations of essays by utilizing the latent variable introduced by variational autoencoder to capture latent semantics information of essays. One of the main advantages of deep neural models is end-to-end training without any hand-crafted feature engineering in traditional discrete models. Besides, neural feature representa-tions tend to be more abstract and high-level compared with discrete manual features. We show that neural feature representations learned by our models outperform traditional discrete models with manual features in the automatic essay scoring task.