Detecting Deepfakes via Robust Spatio-Temporal Residual Learning Using Variational Autoencoders and EfficientNetV2

Deepfakes are a significant threat to digital trust, biometric authentication, and the reliability of online content. Current deepfake detection tools, which are exclusively based on the raw spatial factors are characterized by poor cross-dataset generalization and prone to compression artifacts. To address these shortcomings, this paper presents a robust spatio-temporal residual learning framework to detect deepfakes in images and videos. Variational autoencoder is trained on real images with the adversarial perturbation training using fast gradient sign method (FGSM) and projected gradient descent (PGD) to learn stable and manipulation-sensitive residual maps. In the case of image-based detection, an EfficientNetV2 fine-tuned model is used to classify residual maps. To analyze video, a temporal residual consistency (TRC) module, which is based on a GRU or 1D-CNN model, can be used to capture inter-frame residual inconsistencies. The proposed method achieves 93.8% accuracy and 0.96 AUC on FaceForensics++, 80.1% cross-dataset accuracy on WildDeepFake, and improves video-level detection by 5.1%, demonstrating strong robustness and generalization.