FRAMEWORK PENCEGAHAN SERANGAN DEEPFAKE DAN VOICE PHISHING MENGGUNAKAN MULTI-FACTOR BIOMETRIC AUTHENTICATION

Slamet Slamet

Abstract


Serangan deepfake dan voice phishing (vishing) menjadi ancaman baru dalam dunia keamanan siber, terutama dalam konteks identitas digital dan sistem autentikasi berbasis suara serta wajah. Penelitian ini mengusulkan suatu framework pencegahan serangan deepfake dan voice phishing menggunakan pendekatan Multi-Factor Biometric Authentication (MFBA) yang mengintegrasikan Convolutional Neural Network (CNN) untuk deteksi citra wajah palsu dan Bidirectional Long Short-Term Memory (BiLSTM) untuk verifikasi suara. Framework ini didukung oleh mekanisme weighted decision fusion yang menggabungkan hasil autentikasi biometrik dengan faktor non-biometrik seperti OTP (One-Time Password) dan behavioral pattern recognition. Evaluasi dilakukan menggunakan dataset DFDC (Deepfake Detection Challenge) dan ASVspoof 2021, serta data autentikasi internal. Hasil eksperimen menunjukkan akurasi deteksi deepfake sebesar 96,7%, precision 95,2%, recall 97,5%, dan nilai F1 sebesar 96,3%. Penelitian ini memberikan kontribusi terhadap pengembangan sistem autentikasi cerdas berbasis biometrik multimodal yang lebih tahan terhadap serangan manipulasi visual dan suara

Keywords


Deepfake, Voice Phishing, Multi-Factor Authentication, CNN, BiLSTM, Cybersecurity

Full Text:

PDF hlm 135 - 147

References


I. A. Al-Khazraji, S. H., Saleh, H. H., Khalid, A. I., & Mishkhal, “Impact of deepfake technology on social media: Detection, misinformation and societal implications.,” The Eurasia Proceedings of Science Technology Engineering and Mathematics, vol. 23, no. 2, pp. 429–441, 2023.

T. C. K. Jan Kietzmann, Linda W. Lee, Ian P. McCarthy, “Deepfakes: Trick or treat?,” Business Horizons, vol. 63, no. 2, pp. 135–146, 2020, doi: https://doi.org/10.1016/j.bushor.2019.11.006.

A. Dash, J. Ye, and G. Wang, “A Review of Generative Adversarial Networks (GANs) and Its Applications in a Wide Variety of Disciplines: From Medical to Remote Sensing,” IEEE Access, vol. 12, no. October 2023, pp. 18330–18357, 2024, doi: 10.1109/ACCESS.2023.3346273.

Y. Kang, W. Kim, S. Lim, H. Kim, and H. Seo, “DeepDetection: Privacy-Enhanced Deep Voice Detection and User Authentication for Preventing Voice Phishing,” Applied Sciences (Switzerland), vol. 12, no. 21, 2022, doi: 10.3390/app122111109.

Q. Le Roux, E. Bourbao, Y. Teglia, and K. Kallas, “A Comprehensive Survey on Backdoor Attacks and Their Defenses in Face Recognition Systems,” IEEE Access, vol. 12, no. April, pp. 47433–47468, 2024, doi: 10.1109/ACCESS.2024.3382584.

B. Yan, J. Lan, and Z. Yan, “Backdoor Attacks against Voice Recognition Systems: A Survey,” ACM Computing Surveys, vol. 57, no. 3, 2024, doi: 10.1145/3701985.

S. Ali, S. U. Rehman, A. Imran, G. Adeem, Z. Iqbal, and K. Il Kim, “Comparative Evaluation of AI-Based Techniques for Zero-Day Attacks Detection,” Electronics (Switzerland), vol. 11, no. 23, pp. 1–25, 2022, doi: 10.3390/electronics11233934.

T. Sowmya and E. A. Mary Anita, “A comprehensive review of AI based intrusion detection system,” Measurement: Sensors, vol. 28, no. May, p. 100827, 2023, doi: 10.1016/j.measen.2023.100827.

S. Slamet, “Pertahanan Pencegahan Serangan Social Engineering Menggunakan Two Factor Authentication (2Fa) Berbasis Sms (Short Message System),” Spirit, vol. 14, no. 2, pp. 23–29, 2023, doi: 10.53567/spirit.v14i2.260.

S. Slamet, “Desain Arsitektur Aplikasi Qr Code Sebagai Anti Phishing Serangan Qr Code,” Spirit, vol. 15, no. 1, pp. 42–48, 2023, doi: 10.53567/spirit.v15i1.280.

E. Marasco, M. Albanese, V. V. R. Patibandla, A. Vurity, and S. S. Sriram, “ Biometric multi‐factor authentication: On the usability of the FingerPIN scheme ,” Security and Privacy, vol. 6, no. 1, pp. 1–14, 2023, doi: 10.1002/spy2.261.

M. M. Taye, “Theoretical Understanding of Convolutional Neural Network: Concepts, Architectures, Applications, Future Directions,” Computation, vol. 11, no. 3, 2023, doi: 10.3390/computation11030052.

S. Siami-Namini, N. Tavakoli, and A. S. Namin, “The Performance of LSTM and BiLSTM in Forecasting Time Series,” Proceedings - 2019 IEEE International Conference on Big Data, Big Data 2019, pp. 3285–3292, 2019, doi: 10.1109/BigData47090.2019.9005997.

D. E. Kurniawan, M. Iqbal, J. Friadi, F. Hidayat, and R. D. Permatasari, “Login Security Using One Time Password (OTP) Application with Encryption Algorithm Performance,” Journal of Physics: Conference Series, vol. 1783, no. 1, 2021, doi: 10.1088/1742-6596/1783/1/012041.

R. Venkatesan, S. Shirly, M. Selvarathi, and T. J. Jebaseeli, “Human Emotion Detection Using DeepFace and Artificial Intelligence †,” Engineering Proceedings, vol. 59, no. 1, 2023, doi: 10.3390/engproc2023059037.

C. Korgialas, C. Kotropoulos, and K. N. Plataniotis, “Leveraging Electric Network Frequency Estimation for Audio Authentication,” IEEE Access, vol. 12, no. December 2023, pp. 9308–9320, 2024, doi: 10.1109/ACCESS.2024.3354053.

J. Chen, L. Cai, Y. Tu, R. Dong, D. An, and B. Zhang, “An Identity Authentication Method Based on Multi-modal Feature Fusion,” Journal of Physics: Conference Series, vol. 1883, no. 1, 2021, doi: 10.1088/1742-6596/1883/1/012060.

M. I. Ardiawan and G. P. K. Negarara, “A Comparative Analysis of FaceNet, VGGFace, and GhostFaceNets Face Recognition Algorithms For Potential Criminal Suspect Identification,” Journal of Applied Artificial Intelligence, vol. 5, no. 2, pp. 34–49, 2024.

Z. K. Abdul and A. K. Al-Talabani, “Mel Frequency Cepstral Coefficient and its Applications: A Review,” IEEE Access, vol. 10, no. November, pp. 122136–122158, 2022, doi: 10.1109/ACCESS.2022.3223444.

S. Serrano, L. Patanè, O. Serghini, and M. Scarpa, “Detection and Classification of Obstructive Sleep Apnea Using Audio Spectrogram Analysis,” Electronics (Switzerland), vol. 13, no. 13, pp. 1–27, 2024, doi: 10.3390/electronics13132567.

A. Malik, M. Kuribayashi, S. M. Abdullahi, and A. N. Khan, “DeepFake Detection for Human Face Images and Videos: A Survey,” IEEE Access, vol. 10, pp. 18757–18775, 2022, doi: 10.1109/ACCESS.2022.3151186.

B. Yasser et al., “Deepfake Detection Using EfficientNet and XceptionNet,” Proceedings - 11th IEEE International Conference on Intelligent Computing and Information Systems, ICICIS 2023, no. June, pp. 598–603, 2023, doi: 10.1109/ICICIS58388.2023.10391114.

A. Onan, “Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification,” Journal of King Saud University - Computer and Information Sciences, vol. 34, no. 5, pp. 2098–2117, 2022, doi: 10.1016/j.jksuci.2022.02.025.

B. Dolhansky et al., “The DeepFake Detection Challenge (DFDC) Dataset,” 2020, [Online]. Available: http://arxiv.org/abs/2006.07397.

X. Liu et al., “ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild,” IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 31, pp. 2507–2522, 2023, doi: 10.1109/TASLP.2023.3285283.

H. Sun et al., “An Improved Medical Image Classification Algorithm Based on Adam Optimizer,” Mathematics, vol. 12, no. 16, pp. 1–14, 2024, doi: 10.3390/math12162509.




DOI: http://dx.doi.org/10.53567/spirit.v17i2.406

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Slamet Slamet

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

 

Diindeks Oleh:

SPIRIT : Sarana Penunjang Informasi Terkini

Diterbitkan oleh Teknologi Informasi Institut Teknologi dan Bisnis Yadika Pasuruan
Alamat Redaksi: Jl. Bader No.9, Kwangsan, Kalirejo, Kec. Bangil, Pasuruan, Jawa Timur 67153
Telp/Fax: (0343) 742070 , Email : lppm@stmik-yadika.ac.id
Google Maps :  Klik Disini

 Creative Commons License
Karya ini dilisensikan di bawah  Lisensi Internasional Creative Commons Atribusi 4.0 .