Evaluating SVM and Naïve Bayes Classifiers with Resampling Techniques for Sentiment Analysis of Indonesian PayLater Users on Social Media

Abstrak

Advances in financial technology have popularized PayLater services in Indonesia, enabling online consumers to pay in installments.While offering convenience, these services also pose challenges such as impulsive behavior and default risks.This study analyzes user sentiment toward PayLater by comparing the performance of Support Vector Machine (SVM) and Naï ve Bayes Classifier (NBC) algorithms with resampling techniques.Data were collected from social media, preprocessed, and partially labeled using semisupervised learning.Tweets were classified as positive (support or satisfaction) or negative (criticism or complaint).Class imbalance was addressed using Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic Sampling (ADASYN), and hyperparameters were optimized via GridSearch.Model performance was evaluated with 10-fold cross-validation, considering accuracy, precision, recall, and F1-score.Results indicate that SVM combined with SMOTE and GridSearch achieved the highest accuracy (94.31%) and balanced performance across all metrics.Comparisons show that semisupervised labeling improves model generalization compared to manually labeled data.Visual analyses, including word clouds and sentiment distribution, provide insights into dominant user concerns and satisfaction patterns.This research demonstrates the effectiveness of combining SVM, Naï ve Bayes, and resampling strategies for sentiment classification in imbalanced social media datasets.Findings can guide fintech companies, regulators, and researchers in monitoring public sentiment, improving user engagement, and informing policy development.The study also illustrates a practical deployment workflow for integrating predictive sentiment analysis with web-based applications.

Identifier

• DOI: https://doi.org/10.18280/isi.310402
• OpenAlex: https://openalex.org/W7163418515