Neural networks for text matching

  • Chunlin Xu

Student thesis: Doctoral Thesis

Abstract

Text matching is a task of comparing two texts to identify their relationship. It is an important component of a variety of natural language processing (NLP) tasks. Recently, neural networks provide a new paradigm for text matching. In this thesis, we seek to improve text matching in the following two perspectives: (1) text representation and (2) text interaction.

Tree-Structured Long Short-Term Memory (TreeLSTM) has shown its effectiveness in text representation. To further improve the expressive power of TreeLSTM, we propose two new text representation models: TreeLSTM with Tag-aware Hypernetwork (TagHyperTreeLSTM) and Tag-Enhanced Dynamic Compositional Neural Network (TE_DCNN), respectively. These two models are both devised to alleviate the inability of distinguishing different syntactic compositions of standard TreeLSTM with the aid of Part-of-Speech (POS) tags. TagHyperTreeLSTM contains two separate TreeLSTMs, a tag-aware hypernetwork TreeLSTM to generate parameters of the sentence encoder TreeLSTM dynamically, and a sentence encoder TreeLSTM to generate the final sentence representation. TE_DCNN shares similar framework with TagHyperTreeLSTM, but it extends the standard TreeLSTM with binarized constituency tree to a novel structure, named ARTreeLSTM with general constituency tree in which non-leaf nodes can have any number of child nodes.

In addition, the ability of capturing matching features between two texts is of great significance for improving text matching performance. Two new interaction-based matching models are proposed in this thesis: Multi-Level Compare-Aggregate model (MLCA) and Multi-Level Matching Network (MMN). MLCA matches each word in one text against the other text at three different levels of granularity – word level (word-by-word matching), phrase level (word-by-phrase matching) and sentence level (word-by-sentence matching).MMN utilises multiple levels of word representations such as word embedding, contextualized word representation, to obtain multiple word level matching results for final text level matching decision.

In summary, this thesis proposes four novel neural network based models for text matching. As text matching is a fundamental operation in various NLP tasks and applications including paraphrase identification, natural language inference, machine comprehension, information retrieval and so on, we believe that models presented in this thesis would promote the performance of the above NLP tasks and applications containing text matching. Moreover, text representation models described in this thesis would also be helpful for some other NLP applications such as text classification.
Date of AwardApr 2021
Original languageEnglish
SupervisorHui Wang (Supervisor), Shengli Wu (Supervisor) & Zhiwei Lin (Supervisor)

Keywords

  • Text matching
  • Text classification
  • Natural language processing

Cite this

'