Voice Deepfake Detection Using Spectrogram Features and Mel-Frequency Cepstral Coefficients with a Combination of Convolutional Neural Network and Recurrent Neural Network

Abstrak

The rapid development of artificial intelligence technology has driven the emergence of voice deepfakes that are increasingly realistic and difficult to distinguish from genuine human speech. This condition creates significant risks, including identity misuse, fraud, and information manipulation, thereby requiring an effective detection system capable of identifying manipulated voice patterns with high accuracy. This study aims to develop a voice deepfake detection system by combining Convolutional Neural Network (CNN) and Recurrent Neural Network (BiLSTM) methods. Audio features were extracted using Mel-Frequency Cepstral Coefficients (MFCC) and spectrograms to capture both frequency characteristics and temporal dynamics simultaneously. All genuine and deepfake voice recordings underwent preprocessing stages, including noise reduction, normalization, segmentation, and augmentation, to improve data diversity and model robustness. The model was evaluated using several data split ratios to determine the most optimal performance. The best result was achieved with an 80:20 ratio, reaching an accuracy of 99.3% and an AUC value of 0.9999. These results demonstrate the model’s strong capability to identify subtle structural changes in audio signals that are difficult to detect using conventional methods. Based on these findings, the CNN–RNN (BiLSTM) approach proved to be highly effective for detecting manipulated voice recordings. This research provides an important contribution to the development of audio security systems and the mitigation of risks associated with the misuse of deepfake technology across various sectors.

Identifier

• DOI: https://doi.org/10.59934/jaiea.v5i3.2507
• OpenAlex: https://openalex.org/W7167507111