Integrating Whisper Transcription with Transformer and CNN Models for Indonesian Abusive Speech Detection

Abstrak

The growing popularity of audio social media in Indonesia has fueled the spread of profanity and hate speech, even though there are no regulations specifically covering its use. In this work, we propose an audio-based coarse word detection model by combining one Whisper model for transcription and another IndoBERT in a sentence context classifier to decode it along with a Convolutional Neural Network (CNN) for fine-grained coarse word prediction. The development of IDSS was based on a Knowledge Discovery in Database (KDD) process, consisting of the data selection, preprocessing, transformation, data mining and evaluation phases. The recovered dataset will be pre-processed through cleaning, tokenizing and stemming accordingly prior to labeling. To evaluate the data sharing strategies we used three proportions of shared information (80:20, 70:30 and 60:40) with 5-fold cross-validation. It was found that scenario 1 having data sharing ratio of 80:20 gave highest performance with 100% training accuracy, 98.71% validation accuracy and 99.57% test accuracy. The fusion of IndoBERT and CNN seems to be effective in finding broad sentences and certain words including rudeness. The proposed system would be a first line of defense in reducing the dissemination of hateful language content on auditory social media.

Identifier

• OpenAlex: https://openalex.org/W7169873558
• DOI: https://doi.org/10.67561/jaids.v1i2.1