Sentiment Analysis of Mamikos User Reviews Using FastText and Text Convolutional Neural Network
Abstrak
With the number of applications offering similar services, service providers are required to compete in developing and maintaining well-performing services. One effective approach to achieving this is through sentiment analysis of user reviews collected from the Google Play Store and Apple App Store to identify user-related issues. Accordingly, this study aims to analyze user sentiment toward the Mamikos application using the FastText and Text Convolutional Neural Network (TextCNN). The dataset used in this study consists of user reviews with an imbalanced class distribution between positive and negative sentiments, and the labeling process is conducted by a single annotator, which may introduce potential bias. Before model construction, the dataset undergoes preprocessing and word embedding using FastText. Subsequently, the TextCNN method is employed to develop a sentiment classification model that categorizes user reviews into positive and negative sentiment classes. The performance of the proposed model is evaluated using accuracy, precision, recall, f1-score, and receiver operating characteristic – area under the curve (ROC-AUC). This study makes two contributions. First, it empirically evaluates the combination of FastText and TextCNN on Indonesian user review data with an imbalanced class distribution. Second, it incorporates ROC-AUC as a complementary metric to provide a more comprehensive assessment than accuracy-based measures alone. The experimental results demonstrate that the model achieves an accuracy of 95.14%, a recall of 94.05%, an F1 score of 94.31% and an ROC-AUC score of 0.9883. These results suggest that the model is effective at classifying sentiment and distinguishing between classes. This study makes a valuable contribution to the field of natural language processing by demonstrating the effectiveness of computationally efficient deep learning models for handling imbalanced, domain-specific text classification tasks.
Identifier
- DOI: https://doi.org/10.59653/ijmars.v4i02.2540
- OpenAlex: https://openalex.org/W7166352544