Klasifikasi Ujaran Kebencian Menggunakan 5-Fold Ensemble dengan Weighted Probability Averaging pada Arsitektur Twitter-RoBERTa

Abstrak

Social media platforms have become a critical medium for hate speech propagation at unprecedented scale, with over 66.8 million user reports regarding hateful conduct recorded on platform X during the first half of 2024 alone. This study proposes an end-to-end NLP pipeline for automated hate speech classification using the domain-adapted Twitter-RoBERTa architecture, evaluated on the HASOC (Hate Speech and Offensive Content Identification) English datasets from 2020 and 2021. The core challenge addressed is Transformer fine-tuning instability on relatively small annotated corpora caused by extreme sensitivity to random seed initialization and suboptimal hyperparameter configurations. Three methodological innovations are synergistically integrated: (1) Bayesian Optimization via the Optuna framework for automated adaptive hyperparameter search with 15 trials; (2) Stratified 5-Fold Cross-Validation for robust, reproducible data partitioning; and (3) Weighted Probability Averaging (WPA) as the ensemble aggregation strategy. Results demonstrate that the proposed architecture achieves a Macro F1-Score of 80.99% on Subtask 1A and 64.70% on Subtask 1B, positioning it competitively against 65 international research teams on the official HASOC 2021 leaderboard.

Identifier

• DOI: https://doi.org/10.47065/bulletinds.v5i3.10145
• OpenAlex: https://openalex.org/W7166709141