Abstract
Capturing the similarities between human language units is crucial for explaining how humans associate different objects, and therefore its computation has received extensive attention, research, and applications. With the ever-increasing amount of information around us, calculating similarity becomes increasingly complex, especially in many cases, such as legal or medical affairs, measuring similarity requires extra care and precision, as small acts within a language unit can have significant real-world effects. My research goal in this thesis is to develop regression models that account for similarities between language units in a more refined way. Computation of similarity has come a long way, but approaches to debugging the measures are often based on continually fitting human judgment values. To this end, my goal is to develop an algorithm that precisely catches loopholes in a similarity calculation. Furthermore, most methods have vague definitions of the similarities they compute and are often difficult to interpret. The proposed framework addresses both shortcomings. Itconstantly improves the model through catching different loopholes. In addition, every refinement of the model provides a reasonable explanation. The regression model introduced in this thesis is called progressively refined similarity computation, which combines attack testing with adversarial training. The similarity regression model of this thesis achieves state-of-the-art performance in handling edge cases. Chapter 2 is an introductory chapter on similarity computation in general. The main four chapters of the thesis explore the applications of general similarities, how to capture their omissions, and how to design a similarity model that can be refined over time. The first practical work looks at applying similarity classify topics in online discussions on social networking services such as Twitter. The popularity of these services, and the large number of tweet, challenges the automatic topic detection models. To complicate matters, these topics need to be identified in the absence of prior knowledge about their type and number, and expertise is needed to tune numerous parameters. To address this challenge, I modified the cluster-based topic modelling algorithm that is originally based on word networks and n-grams co-occurrence. A more stable similarity scheme based on word embedding is used to construct networks, and the refined algorithm can better utilise community detection methods to determine topics. Similarity so far takes a binary value, despite its regression value being calculated. Only a medium level of precision is required to determine whether similar or not. However, regression model requires higher precision. To test similarity schemes, I next construct a special task, "revising for concision", stemming from an academic writing need, which is to write concisely to convey meaning clearly. I curate a parallel dataset that describes concise revisions through 536 pairs of sentences before and after revision. These syntactically distinct but semantically identical sentences have a unique role in detecting the type of similarity that a scheme actually captures. I find that the precise semantics are still elusive, and lexical similarity is more frequently captured. Meanwhile, I use this dataset for language generation tasks to help writers struggling to revise drafts. I formulate revision as a constrained paraphrase generation task, where an algorithm is required to rewrite the sentence with only the necessary words, while preserving its meaning. After stress testing these regression models using ready-made sentences, I define attacks to the regression models. I choose the scoring system of automatic summarization to be attacked. The automatic scoring of summaries guides the development of summarizers and involves aspects such as fluency, grammar, and even textual entailment. In this work, I perform evasion attacks to explore robustness of scoring systems. Attack systems predict a non-summary string from each input, and these non-summary strings achieve competitive similarity scores on representative metrics, even outperforming state-of-the-art summarization methods, indicating the low robustness of current similarity schemes. Backdoor attacks are also performed. In a final work, I explore the indistinct nature of similarity. I focus on semantic similarity, a well-known interpretation for which is natural language inference. But, there is little community consensus on its nature. Disagreements are reflected in deciding whether the inference is based on semantics or pragmatics, the strength and form of the inference, and the relevance of the inference to the real world. Differing views on the nature of inferences can further lead to differing views on whether a particular inference holds. Therefore, a system may face difficulties in evaluating reasonably under different perspectives. I propose an interpretation of the inference that takes advantage of predicate logic, natural deduction, and conditional probability to resolve seemingly inconsistencies between different viewpoints. I conclude this thesis by analysing the overall issues plaguing similarity research, the pros and cons of my approach, and challenges for future work. Computing similarity is a widely-applied and philosophically profound task, and I hope this thesis benefits future studies.