Na Li, Cheng Li, Honglie Zhang, Journal of Information Processing Systems Vol. 19, No. 5, pp. 631-640, Oct. 2023  

https://doi.org/10.3745/JIPS.02.0205
Keywords: Keyword Overlap, Lexical Chunk Theory, Short English Text, Similarity Algorithm
Fulltext: PDF Full Text PubReader

Abstract

Short-text similarity calculation is one of the hot issues in natural language processing research. The conventional keyword-overlap similarity algorithms merely consider the lexical item information and neglect the effect of the word order. And some of its optimized algorithms combine the word order, but the weights are hard to be determined. In the paper, viewing the keyword-overlap similarity algorithm, the short English text similarity algorithm based on lexical chunk theory (LC-SETSA) is proposed, which introduces the lexical chunk theory existing in cognitive psychology category into the short English text similarity calculation for the first time. The lexical chunks are applied to segment short English texts, and the segmentation results demonstrate the semantic connotation and the fixed word order of the lexical chunks, and then the overlap similarity of the lexical chunks is calculated accordingly. Finally, the comparative experiments are carried out, and the experimental results prove that the proposed algorithm of the paper is feasible, stable, and effective to a large extent.

Statistics
Cite this article