Tag-Enhanced Dynamic Compositional Neural Network over Arbitrary Tree Structure for Sentence Representation

Chunlin Xu, Hui Wang, Shengli Wu, Zhiwei Lin

Research output: Contribution to journalArticlepeer-review

Abstract

Learning the distributed representation of a sentence is a fundamental operation for a variety of natural language processing tasks, such as text classification, machine translation, and text semantic matching. Tree-structured dynamic compositional networks have achieved promising performance in sentence representation due to its ability in capturing the richness of compositionality. However, existing dynamic compositional networks are mostly based on binarized constituency trees which cannot represent the inherent structural information of sentences effectively. Moreover, syntactic tag information, which is demonstrated to be useful in sentence representation, has been rarely exploited in existing dynamic compositional models. In this paper, a novel LSTM structure, ARTree-LSTM, is proposed to handle general constituency trees in which each non-leaf node can have any number of child nodes. Based on ARTree-LSTM, a novel network model, Tag-Enhanced Dynamic Compositional Neural Network (TE-DCNN), is proposed for sentence representation learning, which contains two ARTree-LSTMs, i.e. tag-level ARTree-LSTM and word-level ARTree-LSTM. The tag-level ARTree-LSTM guides the word-level ARTree-LSTM in conducting dynamic composition. Extensive experiments demonstrate that the proposed TE-DCNN achieves state-of-the-art performance on text classification and text semantic matching tasks.
Original languageEnglish
Number of pages23
JournalExpert Systems with Applications
Publication statusAccepted/In press - 3 Apr 2021

Fingerprint Dive into the research topics of 'Tag-Enhanced Dynamic Compositional Neural Network over Arbitrary Tree Structure for Sentence Representation'. Together they form a unique fingerprint.

Cite this