Feature Analysis for Detecting Mobile Application Review Generated by AI-Based Language Model


Seung-Cheol Lee, Yonghun Jang, Chang-Hyeon Park, Yeong-Seok Seo, Journal of Information Processing Systems Vol. 18, No. 5, pp. 650-664, Oct. 2022  

10.3745/JIPS.02.0182
Keywords: Artificial intelligence, Fake Review, GPT-2, Language Model, Machine Learning, software engineering
Fulltext:

Abstract

Mobile applications can be easily downloaded and installed via markets. However, malware and malicious applications containing unwanted advertisements exist in these application markets. Therefore, smartphone users install applications with reference to the application review to avoid such malicious applications. An application review typically comprises contents for evaluation; however, a false review with a specific purpose can be included. Such false reviews are known as fake reviews, and they can be generated using artificial intelligence (AI)-based text-generating models. Recently, AI-based text-generating models have been developed rapidly and demonstrate high-quality generated texts. Herein, we analyze the features of fake reviews generated from Generative Pre-Training-2 (GPT-2), an AI-based text-generating model and create a model to detect those fake reviews. First, we collect a real human-written application review from Kaggle. Subsequently, we identify features of the fake review using natural language processing and statistical analysis. Next, we generate fake review detection models using five types of machine-learning models trained using identified features. In terms of the performances of the fake review detection models, we achieved average F1-scores of 0.738, 0.723, and 0.730 for the fake review, real review, and overall classifications, respectively.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from November 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.




Cite this article
[APA Style]
Lee, S., Jang, Y., Park, C., & Seo, Y. (2022). Feature Analysis for Detecting Mobile Application Review Generated by AI-Based Language Model. Journal of Information Processing Systems, 18(5), 650-664. DOI: 10.3745/JIPS.02.0182.

[IEEE Style]
S. Lee, Y. Jang, C. Park, Y. Seo, "Feature Analysis for Detecting Mobile Application Review Generated by AI-Based Language Model," Journal of Information Processing Systems, vol. 18, no. 5, pp. 650-664, 2022. DOI: 10.3745/JIPS.02.0182.

[ACM Style]
Seung-Cheol Lee, Yonghun Jang, Chang-Hyeon Park, and Yeong-Seok Seo. 2022. Feature Analysis for Detecting Mobile Application Review Generated by AI-Based Language Model. Journal of Information Processing Systems, 18, 5, (2022), 650-664. DOI: 10.3745/JIPS.02.0182.