SupCon-MPL-DP: Supervised Contrastive Learning with Meta Pseudo Labels for Deepfake Image Detection

Abstract

Recently, there has been considerable research on deepfake detection. However, most existing methods face challenges in adapting to the advancements in new generative models within unknown domains. In addition, the emergence of new generative models capable of producing and editing high-quality images, such as diffusion, consistency, and LCM, poses a challenge for traditional deepfake training models. These advancements highlight the need for adapting and evolving existing deepfake detection techniques to effectively counter the threats posed by sophisticated image manipulation technologies. In this paper, our objective is to detect deepfake videos in unknown domains using unlabeled data. Specifically, our proposed approach employs Meta Pseudo Labels (MPL) with supervised contrastive learning, so-called SupCon-MPL, allowing the model to be trained on unlabeled images. MPL involves the simultaneous training of both a teacher model and a student model, where the teacher model generates pseudo labels utilized to train the student model. This method aims to enhance the adaptability and robustness of deepfake detection systems against emerging unknown domains. Supervised contrastive learning utilizes labels to compare samples within similar classes more intensively, while encouraging greater distinction from samples in dissimilar classes. This facilitates the learning of features in a diverse set of deepfake images by the model, consequently contributing to the performance of deepfake detection in unknown domains. When utilizing the ResNet50 model as the backbone, SupCon-MPL exhibited an improvement of 1.58% in accuracy compared with traditional MPL in known domain detection. Moreover, in the same generation of unknown domain detection, there was a 1.32% accuracy enhancement, while in the detection of post-generation unknown domains, there was an 8.74% increase in accuracy.

1. Introduction

Recently, with the advancement of generative artificial intelligence models [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15], deepfakes have become increasingly similar to real images/videos, making them difficult to distinguish. Deepfakes can be broadly categorized into three generations based on the evolution of generative models. First-generation deepfake generative models [1,2,3,4,5] typically attempt to generate simple and low-resolution images/videos based on probability distributions or synthesize multiple images/videos by exploiting features in tasks such as Face2Face and FaceSwap. In particular, generative adversarial network (GAN)-based models such as CGAN [2], WGAN [4], and WGAN-GP [5], as well as autoencoder-based models like VAE [6] and conditional VAE [7] have enabled the generation of various deepfake images/videos. However, first-generation deepfakes often exhibit noticeable artifacts that can be discerned by the human eye. With the transition to second-generation deepfake generative models [6,7,8], there has been progress in generating high-resolution deepfake images that are more difficult to distinguish compared with first-generation ones, along with performance improvements in various tasks. In particular, second-generation deepfake generative models like StyleGAN, proposed by T. Karras et al. [8], produce deepfake images that are difficult for the human eye to distinguish, excluding some flaws such as artifacts in hair. Second-generation deepfakes can be generated using deepfake generation tools such as DeepfaceLab [10], DeepSwap [11], Synthesia [12], and others. Finally, third-generation deepfake generative models [9,13,14,15] produce images/videos that are even more flexible and difficult to distinguish than those generated by second-generation models, across various tasks. In particular, Stable Diffusion, proposed by R. Rombach et al. [13], is currently being used for the generation of various human and artwork images, raising concerns related to copyright and human rights issues. Furthermore, the consistency model proposed by Y. Song et al. [15] has enabled state-of-the-art deepfake generative model training at a lower cost by reducing the extensive iteration process required by previous diffusion models for restoring original images from noise. As deepfakes increasingly become difficult to distinguish from real images/videos, they are being utilized in various criminal activities.

To address issues caused by deepfakes, methods have been proposed to identify flaws in landmarks that occur when deepfake generative models create images, aiding in the detection of deepfakes [16,17,18,19]. Meanwhile, recent advancements in deepfake detection for single models have shown improvement in detecting deepfake videos and images. D.A. Coccomini et al. [20] enhanced the detection performance of deepfake videos and images by combining EfficientNet [21] and Vision Transformer [22] when training a single model. In other words, existing deepfake detection models [16,17,18,19,20,21,22,23] verify flaws in facial landmarks during the preprocessing stage and construct large models for flexible predictions.

Previous studies have primarily focused on the detection performance of labeled known domain (known domain) tasks in deepfake detection. However, deepfake generation models are rapidly evolving, and similar generations of deepfake generation models are also being developed diversely. Therefore, detecting deepfake images in unknown domains (unknown domain) is also crucial. A few studies have proposed generalized deepfake detection models using techniques such as contrastive learning, meta learning, and others [24,25,26,27,28,29,30,31,32,33,34,35,36].

In this paper, we propose SupCon-MPL, a combination of the Meta Pseudo Labels (MPL) [37] with supervised contrastive learning (SupCon) [38] to further train the model with unlabeled images/videos, simultaneously enhancing the model’s generalization ability to distinguish deepfakes in unknown domains. The proposed SupCon-MPL utilizes the basic structure of MPL, where two models, namely teacher and student, are simultaneously trained. Each model influences the other during training. The teacher model constructs pseudo labels for unlabeled images and transfers them to the student model. Through this approach, the student model learns from unlabeled data, providing the potential to train effectively with limited labeled data. Furthermore, during the training process, we apply the supervised contrastive loss (SupConLoss) [38] to the encoder of each model, enabling contrastive representation learning, thereby inducing generalized model training.

The performance evaluation experiments were conducted in two parts: model validation experiments and deepfake detection experiment based on scenario. In the model validation experiments, we utilized the data from five domains within FaceForensics++ [39]. We evaluated the detection performance in labeled known domains by combining the data in various ways and assessed the generalized detection performance in unknown domains. The deepfake detection experiment based on scenario involves training the model with first-generation deepfake datasets (FaceForensics++ [39], DFDC [40], Celeb-DF [41]) and evaluating the detection performance on first- and second-generation unknown deepfake datasets (StyleGAN [8], NeuralTextures [39]). The experimental results showed that SupCon-MPL achieved performance improvements of 1.58%, 1.32%, and 8.74% over the baseline MPL model in the proposed evaluation scenario, respectively. The main contributions of this paper are as follows:

(1)

The proposed method enables additional training through unlabeled data. Especially, while two models are trained simultaneously, the student model infers information about unlabeled data from the other model and provides feedback, allowing for additional training to be conducted with less bias towards a specific model. Ultimately, the proposed method enhances the performance of deepfake detection by enabling additional training with a large amount of unlabeled data.

(2)

Our model enables generalized deepfake detection model training through contrastive learning. We improved the generalized deepfake detection performance on unknown data, which was previously low in the Meta Pseudo Labels-based deepfake detection model [24], through contrastive learning.

(3)

Our model exhibited higher deepfake detection performance compared with all other models in the comparison with various generalized deepfake detection models [24,31,34,35]. The experimental results demonstrate that our model outperforms existing deepfake detection models, showing robust detection capability across diverse labeled datasets and even unknown generational deepfakes.

2. Related Works

As deepfake generation models advance, various detection methods have also been researched. A common approach in deepfake detection is to explore flaws in facial images [16,17,18,19]. However, the continual development of new generative models has led to the problem of being unable to train detection models using data from all generative models. To address this problem, a few studies have explored training generalized deepfake detection models [24,25,26,27,28,29,30,31,32,33,34,35,36].

2.1. Generalized Deepfake Detection

The generalization of deepfake detection implies the ability to detect deepfake videos generated not only by the models used during training but also by unseen or new generative models. In other word, as generative models progress from GANs and VAEs to diffusion and consistency models, achieving the ability to detect deepfakes generated by various and new generative models simultaneously is the main goal of generalized deepfake detection techniques. Recently, research has been conducted on detecting deepfake videos that are unknown from both the data and training perspectives.

On the data perspective, SBL [29] and OST [30] enhanced the generalization of deepfake detection by synthesizing additional training data by combining original images from each generative model with various other images and selectively using them. On the training perspective, A. Jain et al. [25] utilized datasets from Google, Jigsaw, FaceForensics++ [39], Celeb-DF [41], Deepfake-TIMIT [42], and their own database DF-Mobio to train a generalized deepfake detection model using contrastive representation learning across various domains. A. Nadimpalli et al. [26] proposed a hybrid learning technique combining supervised learning and reinforcement learning. In particular, during the training process, the reinforcement learning agent selects the top k augmentations that have the most significant impact on performance improvement when training through convolutional neural networks (CNN) and uses them for testing, enabling the training of a generalized deepfake detection model. We employed the meta-learning technique, Meta Pseudo Labels, in the deepfake training process, applying it after domain splitting for each data, resulting in training a model with higher performance in the same model training [24], and conducted experiments using various CNN-based models specialized in extracting features from images, including EfficientNet [21], ResNet [43], ResNext [44], and WideResNet [45], which are image classification models.

2.2. Contrastive Representation Learning in Deepfake Detection

Currently, research applying contrastive representation learning (CRL) for training generalized deepfake detection models is conducted. CRL enables learning similar features in the feature space between a specific image and from the same domain (positive images), while also learning features that differentiate from different domains (negative images). H. Chih-Chung et al. [27] utilized contrastive loss [43] to train the encoder, following which they trained the classifier to generalize the discriminative performance on deepfake images generated by various GAN-based models. S. Fung et al. [36] trained the encoder using unsupervised CRL with image pairs that include random augmentation applied to the same image during the training process. Following this, they trained the classifier using labeled images to develop a generalized deepfake detection model. X. Ying et al. [35] addressed the issue of conventional CRL techniques not utilizing label information of deepfake images by applying supervised contrastive learning (SupCon) [38], which uses label information. However, CRL requires a large amount of data, especially a significant number of negative samples. In this paper, we performed CRL using SupCon [38], while simultaneously combining the SupCon model with meta-learning method, MPL [37]. This approach also allows for additional CRL training on unlabeled data, even when using the same labeled data, enabling the training of a more generalized deepfake detection model compared with conventional Meta Pseudo Labels.

2.3. Meta Pseudo Labels

Meta Pseudo Labels (MPL) [37] trains the model using unlabeled images, and when the same model is trained on an image classification task, it has shown improved performance compared with conventional models. MPL gained significant attention by achieving over 90% Top 1 score on the ImageNet [44] classification task, marking a significant milestone. Figure 1a shows the MPL facilitates the learning of the teacher model through feedback from the student model, thus enhancing the conventional learning techniques such as knowledge distillation [45] or noisy student [46], where the teacher model passes on information to the student model. This improvement addresses the issue of inadequate learning of the student model when the performance of the teacher model is subpar. The training process of the MPL is shown in Figure 1b. The student model in MPL learns through the Pseudo Labels inferred by the teacher model. Subsequently, it imparts the feedback value regarding the learning to the teacher model. The teacher model learns through the labeled loss from the labeled data, UDA loss [47], feedback from the student model, and MPL loss from the unlabeled data. However, it consumes substantial computing resources due to the simultaneous training of the two models.

From the perspective of training a generalized deepfake detection model, MPL can enhance the performance of generalized deepfake detection by enabling additional training through unlabeled data, compared with models trained solely with labeled data. In this paper, we experiment with the enhancement of detection capabilities for unknown domains and post-generation deepfakes, using both MPL [37] and SupCon [38].

3. Proposed SupCon-MPL-Based Deepfake Detection

To detect deepfake videos in the deepfake unknown domain, the proposed method introduces SupCon-MPL, a meta-learning model based on contrastive learning, utilizing unlabeled images from the deepfake known domain. You can find notations used in the rest of paper summarized in

Table A1. Summary of notations.

Symbol	Definition
CRL	Contrastive Representation Learning.
MPL	Meta Pseudo Labels.
GAN	Generative Adversarial Networks.
VAE	Variational Auto Encoder.
$\mathit{K}$	Known domain which involves a set of known deepfake generative models.
$\mathit{U}$	Unknown domain which involves a set of unknown deepfake generative models.
${\mathit{D}}_{\mathit{K}}$	Deepfake datasets of the known domain which are labeled.
${\mathit{D}}_{\mathit{U}}$	Deepfake datasets of the unknown domain which are unlabeled.
$K_i$	$i$-th known deepfake generative model in known domain.
$U_i$	$i$-th unknown deepfake generative model in unknown domain.
$x_i$	Images of real and deepfake.
$y_i$	Labels of real and deepfake.
$D_L$	Labeled dataset used while training.
$D_{UL}$	Unlabeled dataset used while training.
$T$	Teacher model of SupCon-MPL.
$S$	Student model of SupCon-MPL.
${\theta }_T$	Parameters of the teacher model’s classifier.
${\theta }_S$	Parameters of the student model’s classifier.
${\theta }_S^{PL}$	Minimized parameters of model in SupCon-MPL.
${\mathsf{\theta }}_{T,ENC}^{\left(t\right)}$	Parameters of teacher encoder.
${\theta }_{S,ENC}^{\left(t\right)}$	Parameters of student encoder.
$\mathsf{\eta }s$	Learning rate used while training.
$Cl{F}_T$	Classifier in the teacher model.
$EN{C}_T$	Encoder in the teacher model.
$CL{F}_S$	Classifier in the student model.
$EN{C}_S$	Encoder in the student model.
$x_l$	Labeled images used while training.
$x_u$	Unlabeled images used while training.
$y_l$	Labels used while training.
${\widehat{y}}_u$	Pseudo label generated by the teacher model.
$em{b}_l^T$	Embedding value derived by passing the labeled data through the teacher encoder.
$em{b}_u^S$	Embedding value derived by passing the unlabeled data through the student encoder.
$h$	Student model’s feedback factor.
$I$	Index of the randomly augmented data.
$P\left(i\right)$	Set of indices for all positives in the batch.
${\mathrm{z}}_{\mathrm{i}}, z_p$	Embedding values of each randomly augmented image.

.

3.1. Proposed Training Strategy

3.1.1. Known Domain and Unknown Domain in Deepfake

A deepfake domain can be defined as a collection of images and their features, generated from a “specific deepfake generative model”. In this paper, we distinguish deepfake domains into the known domain ($\mathit{K}$) and unknown domain ($\mathit{U}$). The known domain ($\mathit{K}$) refers to a collection of deepfake images that are labeled when training models. The data in $\mathit{K}$ is labeled and therefore can be directly used for training. Meanwhile, the unknown domain ($\mathit{U}$) refers to data created by unknown deepfake generative models. The data in $\mathit{U}$ is not labeled, hence it is not possible to determine whether the image is real or fake. Also, as they are created from various generative models, they can involve various features. The known domain $\mathit{K}$ can be defined as $\mathit{K}=\{{\mathrm{K}}_1,{\mathrm{K}}_2,\cdots \}$ where $K_i$ is i-th known deepfake generative model, and the deepfake dataset ${\mathit{D}}_{\mathit{K}}=\{D_{K_1},D_{K_2},\cdots \}$ consists of a dataset ${\mathit{D}}_{{\mathit{K}}_{\mathit{i}}}=\left\{\left(x_i,y_i\right)\right\}$ composed with a set of deepfake images $x_i$ and labels $y_i$ generated by $K_i$. On the other hand, the unknown domain $\mathit{U}$ can be defined as $\mathit{U}=\{U_1,U_2,\cdots \}$ where $U_i$ is i-th unknown deepfake generative model, and the deepfake dataset ${\mathit{D}}_{\mathit{U}}=\{D_{U_1},D_{U_2},\cdots \}$ consists of a dataset $D_{U_i}=\left\{\left(x_i\right)\right\}$ composed with a set of deepfake images $x_i$ generated by $U_i$.

In this paper, to address $\mathit{U}$, we first experiment by distinguishing ${\mathit{D}}_{\mathit{K}}$ into a labeled dataset ($D_L$) and an unlabeled dataset ($D_{UL}$), as shown in Figure 2a. Subsequently, to verify the influence of ${\mathit{D}}_{\mathit{U}}$ on the training process, we assume ${\mathit{D}}_{\mathit{U}}$ as $D_{UL}$ and perform experiments, as shown in Figure 2b. In the deepfake training scenario, from the perspective of generative models by generation, both $D_L$ and $D_{UL}$ constitute with first-generation ${\mathit{D}}_{\mathit{K}}$, and evaluation is conducted using the first- and second-generation ${\mathit{D}}_{\mathit{U}}$.

3.1.2. Training Strategy for Deepfake Unknown Domain Detection

The training process is employed based on a comparison between the base model and the student model of the MPL (SupCon-MPL). Upon completion of training the base model with the entire dataset ${\mathit{D}}_{\mathit{K}}$, the model is subsequently employed as the teacher model to train the student model. In other words, we aim to verify performance improvement when training the model under the same conditions. If performance enhancement is validated at this method, it suggests that superior performing models can be trained under identical training conditions, even when employing larger or state-of-the-art (SOTA) models.

The training images are constructed considering the problems of existing deepfake detection. While deepfakes by known generative models exist in $\mathit{K}$, deepfake images by unknown generative models also exist in $\mathit{U}$. Therefore, during the training phase, we enhance the deepfake detection performance in $\mathit{K}$ using labeled data and contribute to the generalization of the learning model by using data ${\mathit{D}}_{\mathit{K}}$ and ${\mathit{D}}_{\mathit{U}}$ from $\mathit{K}$ and $\mathit{U}$ as unlabeled data, respectively. Consistent with this approach, the data $D_L$ and $D_{UL}$ are structured into ${\mathit{D}}_{\mathit{K}}$, with images from dataset ${\mathit{D}}_{\mathit{U}}$ serving as $D_{UL}$.

In the proposed method, we combine data in three strategies to detect the unknown domain dataset ${\mathit{D}}_{\mathit{U}}$. The first strategy is to use the data from $D_L$ and $D_{UL}$ as the same domain, aiming to verify whether unlabeled data from a specific $\mathit{K}$ contributes to the improvement of model performance. Figure 2a illustrates the training strategy of using ${\mathit{D}}_{\mathit{K}}$ as unlabeled data. The second strategy aims to solve the realistic deepfake problem by experimenting with the impact of unlabeled data on the detection performance of the corresponding domain. Figure 2b illustrates the feasibility of improving model performance by employing dataset ${\mathit{D}}_{\mathit{K}}$ as labeled data $D_L$ and dataset ${\mathit{D}}_{\mathit{U}}$ as unlabeled data $D_{UL}$. Finally, in the deepfake scenario experiment, after training the model using the first-generation deepfake dataset as $D_L$ and $D_{UL}$, the generalized deepfake detection model learning is assessed through the first-generation ${\mathit{D}}_{\mathit{K}}$ and the first- and second-generation ${\mathit{D}}_{\mathit{U}}$.

3.2. SupCon-MPL: Supervised Contrastive Learning with Meta Pseudo Labels

In the proposed method, following the strategy in Figure 2b, the MPL model is trained for the detection of deepfakes in the unknown domain $\mathit{U}$. SupCon-MPL allows supplementary training utilizing unlabeled videos, and with the aid of CRL, it enhances the deepfake detection in feature space. Furthermore, it affords the flexibility to employ diverse encoder models during the training phase and enables the fine tuning of the SupCon-MPL-trained model.

In particular, the limitations of deepfake detection with limited labeled data can be mitigated by using unlabeled data, and a generalized detection model can be trained through CRL. Another notable advantage lies in the capability to conduct concurrent learning via feedback from the student model, even if the performance of the T model is low. The details of the proposed method are elucidated in Figure 3. The most significant distinction from the conventional MPL and SupCon model training is that learning through unlabeled data not only resolves the training issue of CRL due to limited data but also enhances detection capabilities in both $\mathit{K}$ and $\mathit{U}$. Ultimately, the final goal is to enhance the detection capabilities of deepfakes in domains that are not targeted, especially in a situation where new deepfake models in $\mathit{U}$ continue to be developed.

Supcon-MPL consists of a teacher model $T$ and a student model $S$. Each model has the same structure but different parameter values. The teacher model ($T$) utilizes a pre-trained model, while the student model ($S$) starts its training from the initial state before being trained. Each model consists of an identical structure of an encoder and a classifier. The encoder is modified to extract a 128-dimensional feature by removing the classifier layer of a specific model, allowing it to primarily learn the representations of labeled and unlabeled images. The classifier is composed of a single linear layer, which performs classification based on the representation values received from the encoder.

The training of the SupCon-MPL is conducted by first having the student model $T$ use unlabeled data to perform CRL, followed by fine tuning with labeled data. In this process, the teacher model $T$’s classifier learns through the feedback from $S$, while $S$ learns dependently on $T$.

3.3. SupCon-MPL Loss Function

SupCon-MPL, as shown in Figure 3, is composed of a teacher model ($T$) and a student model ($S$), each of which consists of an encoder and a linear classifier. SupCon-MPL has two loss functions in order to sequentially train each model. One involves the teacher model $T$ distilling knowledge to the student model $S$, while the other entails the teacher model $T$ training from the feedback factor provided by $S$ on the labeled data. The knowledge distilled by $T$ includes previously learned content about deepfakes.

In SupCon-MPL, let the parameters of $T$ classifier and $S$ classifier be ${\theta }_T$, ${\theta }_S,$ respectively, and denote the batch of images and labels on the labeled data as $\left(x_l,y_l\right)\in D_K$, and the batch of images on the unlabeled data as $x_u\in D_U$. The goal of SupCon-MPL is to minimize the parameters ${\theta }_S^{PL}$ of the generalized deepfake detection model $S$:

$${\theta }_S^{PL}=\underset{{\theta }_S}{\mathit{argmin}}\underset{L_u≔\left({\theta }_T,{\theta }_S\right)}{\underbrace{E_{x_u}\left[CE\left(T\left(x_u;{\theta }_T\right),S\left(x_u;{\theta }_S\right)\right)\right]}}$$

(1)

Hence, the objective function of SupCon-MPL is defined as follows.

$$\begin{array}{c}{L}_l\mathrm{ }\mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h}\mathrm{ }\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{ }\mathrm{t}\mathrm{o}\mathrm{ }{\theta }_T:\\ \underset{{\mathsf{\theta }}_{\mathrm{T}}}{\min }\mathrm{ }{L}_l\left({\theta }_S^{PL}\left({\theta }_T\right)\right),\\ \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{ }{\theta }_S^{PL}\left({\theta }_T\right)=\underset{{\theta }_S}{\mathrm{argmin}}\mathrm{ }{L}_u\left({\theta }_T,{\theta }_S\right).\end{array}$$

(2)

For optimization, SupCon-MPL approximates ${\theta }_S^{PL}\left({\theta }_T\right)$ by the learning rate $\mathsf{\eta }s$, and then,

$${\theta }_S^{PL}\left({\theta }_T\right)\approx {\theta }_S-\eta s\cdot {\nabla }_{{\theta }_S}{L}_u\left({\theta }_T,{\theta }_S\right)$$

(3)

defines the final objective function as follows:

$$\begin{array}{c}{L}_l\mathrm{ }\mathrm{w}\mathrm{i}\mathrm{t}\mathrm{h}\mathrm{ }\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{p}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{ }\mathrm{t}\mathrm{o}\mathrm{ }{\mathsf{\theta }}_T:\\ \underset{{\mathsf{\theta }}_T}{\min }\mathrm{ }{L}_l\left({\theta }_S-\eta s\cdot {\nabla }_{{\theta }_S}{L}_u\left({\theta }_T,{\theta }_S\right)\right),\\ \mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}\mathrm{ }{\theta }_S^{PL}\left({\theta }_T\right)=\underset{{\mathsf{\theta }}_S}{\mathrm{argmin}}\mathrm{ }{L}_u\left({\mathsf{\theta }}_T,{\mathsf{\theta }}_S\right).\end{array}$$

(4)

Both $T$ and $S$ consist of an encoder and a classifier and are trained according to their respective loss functions. The loss function of $EN{C}_T$, the encoder of $T$, is composed of SupConLoss [38], and the loss function of $\mathrm{C}\mathrm{L}{F}_T$, the classifier, is composed of labeled loss for the labeled data $x_l$ and MPL loss, reflecting the feedback from $S$. First and foremost, $g_{T,contrast}^{\left(t\right)}$ the loss function of $EN{C}_T$, receives image pairs $\left(RandAugmen{t}_a\left(x_l\right),RandAugmen{t}_b\left(x_l\right),y_l\right)$ as inputs that reflect different random augmentations on the same image $x_l$ and the label $y_l$. Subsequently, the loss value is obtained by passing image pairs through SupConLoss [38]. At this juncture, given the similarity between the current training process and that of the original MPL’s UDA loss, the utilization of the UDA loss is no longer used:

$$g_{T,contrast}^{\left(t\right)}={\nabla }_{{\theta }_T}SupConLoss\left(RandAugmen{t}_a\left(x_l\right),RandAugmen{t}_b\left(x_l\right),y_l\right){|}_{{\theta }_T={\theta }_T^{\left(t\right)}}$$

(5)

$EN{C}_T$ is promptly updated following the computation of the $g_{T,contrast}^{\left(t\right)}$:

$${\mathsf{\theta }}_{T,ENC}^{\left(t+1\right)}={\mathsf{\theta }}_{T,ENC}^{\left(t\right)}-{\mathsf{\eta }}_S\cdot g_{T,contrast}^{\left(t\right)}$$

(6)

The labeled loss of $Cl{F}_T$, $g_{T,supervised}^{\left(t\right)}$, measures the difference between $y_l$ and the label predicted by $T$ through cross-entropy loss (CE Loss). Here, $em{b}_l^T$ denotes the embedding value derived by passing the labeled data $x_l$ through $EN{C}_T$:

$$g_{T,supervised}^{\left(t\right)}={\nabla }_{{\theta }_T}CE\left(y_l,CL{F}_T\left(em{b}_l^T;{\theta }_T\right)\right){|}_{{\theta }_T={\theta }_T^{\left(t\right)}}$$

(7)

The MPL loss $g_T^{\left(t\right)}$ calculates the difference between the hard pseudo label $y_u$, which is the maximum value extracted from the pseudo labels generated by $T$ through $x_u$, and the logit. Here, $em{b}_u^T$ denotes the embedding value derived by passing the labeled data $x_u$ through $EN{C}_T$:

$$g_T^{\left(t\right)}=h\cdot {\nabla }_{{\theta }_T}CE\left({\widehat{y}}_u,CL{F}_T\left(em{b}_u^T;{\theta }_T\right)\right){|}_{{\theta }_T={\theta }_T^{\left(t\right)}}$$

(8)

The feedback factor $h$ of $S$ was calculated in the same way as the original Meta Pseudo Labels [37], using Taylor expansion to calculate the difference before and after the training of $S$. In the proposed method, we approximated $h$ using the difference from the CE loss value for the labeled data after $S$ was trained to the value before training. This allows the final loss value to converge as the training progresses.

$$h=CE\left(y_l,S\left(x_l;{\theta }_S^{\left(t+1\right)}\right)\right)-CE\left(y_l,S\left(x_l;{\theta }_S^{\left(t\right)}\right)\right)$$

(9)

The final loss function of $Cl{F}_T$ is composed of the sum of each loss function value:

$${\theta }_T^{\left(t+1\right)}={\theta }_T^{\left(t\right)}-\eta s\cdot \left(g_T^{\left(t\right)}+g_{T,supervised}^{\left(t\right)}\right)$$

(10)

$S$ is trained through unlabeled data. The loss function of the student model’s encoder $EN{C}_S$, denoted as $g_{S,contrast}^{\left(t\right)}$, is trained utilizing SupConLoss [38], akin to $EN{C}_T$. It leverages $\left(x_u,{\widehat{y}}_u\right)$, comprising an unlabeled image $x_u$ paired with pseudo labels ${\widehat{y}}_u$, generated by $T$:

$$\begin{array}{c}{g}_{S,contrast}^{\left(t\right)}\hfill \\ ={\nabla }_{{\theta }_T}SupConLoss\left(RandAugmen{t}_a\left(x_u\right),RandAugmen{t}_b\left(x_u\right),{\widehat{y}}_u\right){|}_{{\theta }_S={\theta }_S^{\left(t\right)}}\hfill \end{array}$$

(11)

$EN{C}_S$ is also promptly updated following the computation of the $g_{S,contrast}^{\left(t\right)}$:

$${\theta }_{S,ENC}^{\left(t+1\right)}={\theta }_{S,ENC}^{\left(t\right)}-{\eta }_S\cdot g_{S,contrast}^{\left(t\right)}$$

(12)

The loss function of $CL{F}_S$ is calculated using CE loss for the hard pseudo label ${\widehat{y}}_u$ of $T$ for $x_u$ and the prediction of $S$. Here, $em{b}_u^S$ denotes the embedding value derived by passing the labeled data $x_u$ through $EN{C}_S$:

$${\theta }_S^{\left(t+1\right)}={\theta }_S^{\left(t\right)}-\eta s\cdot {\nabla }_{{\theta }_S}CE\left({\widehat{y}}_u,CL{F}_S\left(em{b}_u^S;{\theta }_S\right)\right){|}_{{\theta }_S={\theta }_S^{\left(t\right)}}$$

(13)

The SupConLoss in $g_{T,contrast}^{\left(t\right)}$, $g_{S,contrast}^{\left(t\right)}$ on the teacher model $T$ and student model $S$ are as follows:

$$SupConLoss=\sum _{i\in I}SupConLos{s}_i=\sum _{i\in I}\frac{-1}{\left|P\left(i\right)\right|}\sum _{p\in P\left(i\right)}log\frac{exp\left(z_i\cdot z_p/\tau \right)}{{\sum }_{a\in A\left(i\right)}exp\left(z_i\cdot z_a/\tau \right)}$$

(14)

Here, $i\in I\equiv \{1,\cdots ,2N\}$ is the index of the randomly augmented data, and $P\left(i\right)\equiv \{p\in A\left(i\right):y_p=y_i\}$ is the set of indices for all positives in the batch (since $y_p$ and $y_i$ are labels of images that have been randomly augmented from $y_l$, they are the same as $y_l$). ${\mathrm{z}}_{\mathrm{i}}$ and $z_p$ are the embedding values of each randomly augmented image in $\left(RandAugmen{t}_a\left(x_l\right),RandAugmen{t}_b\left(x_l\right)\right)$ passed through the encoder $ENC$, and $A\left(i\right)\equiv I\backslash \left\{i\right\}$, and $\mathsf{\tau }$ is the temperature parameter. In other words, the inner product between positive pairs ($i$ and $p$ are the same class but different samples) is maximized through $exp\left(z_i\cdot z_p/\tau \right)$, and the inner product between negative pairs is minimized through $exp\left(z_i\cdot z_a/\tau \right)$, so that the SupConLoss is minimized.

The training process of the proposed SupCon-MPL model for deepfake detection is shown in Algorithm 1.

Algorithm 1 The Deepfake detection method based on SupCon-MPL (Pseudo code)
	Set Labeled data, Unlabeled data with domain splitting [37]. Input: Labeled data $x_l$, $y_l$ and unlabeled data $x_u$. Initialize ${\theta }_T^{\left(0\right)}$ and ${\theta }_S^{\left(0\right)}$. Pretrain Teacher model with $x_l$, $y_l$. For $t=0$ to $N-1$ do
		Sample an unlabeled example $x_u$ and a labeled example $x_l$, $y_l$.
		Sample a pseudo label ${\widehat{y}}_u\sim P\left(\cdot |x_u;{\theta }_T\right)$.
		Compute contrastive loss of student encoder $EN{C}_S$ using the pseudo label ${\widehat{y}}_u$:
		$g_{S,contrast}^{\left(t\right)}={\nabla }_{{\theta }_T}SupConLoss\left(RandAugmen{t}_a\left(x_u\right),RandAugmen{t}_b\left(x_u\right),{\widehat{y}}_u\right){|}_{{\theta }_S={\theta }_S^{\left(t\right)}}$
		Update the student encoder $\mathrm{E}\mathrm{N}{\mathrm{C}}_{\mathrm{S}}$ using the pseudo label $\widehat{y}$:
		${\theta }_{S,ENC}^{(t+1)}={\theta }_{S,ENC}^{\left(t\right)}-{\eta }_S\cdot g_{S, \mathit{\text{contrast}}}^{\left(t\right)}$
		Update the student classifier $CL{F}_S$ using the pseudo label ${\hat{y}}_u$:
		${\theta }_S^{(t+1)}={\theta }_S^{\left(t\right)}-{\left.\eta s\cdot {\nabla }_{{\theta }_S}CE\left({\hat{y}}_u,CL{F}_S\left(em{b}_u^S;{\theta }_S\right)\right)\right|}_{{\theta }_S={\theta }_S^{\left(t\right)}}$
		Compute contrastive loss of teacher encoder $EN{C}_T$ using the labeled data $\left(x_l,y_l\right)$:
		$g_{T,contrast}^{\left(t\right)}={\nabla }_{{\theta }_T}SupConLoss\left(RandAugmen{t}_a\left(x_l\right),RandAugmen{t}_b\left(x_l\right),y_l\right){|}_{{\theta }_T={\theta }_T^{\left(t\right)}}$
		Update the teacher encoder $EN{C}_T$ using the labeled data $\left(x_l,y_l\right)$:
		${\theta }_{T,ENC}^{(t+1)}={\theta }_{T,ENC}^{\left(t\right)}-{\eta }_S\cdot g_{T, \mathit{\text{contrast}} }^{\left(t\right)}$
		Compute gradient on labeled data $\left(x_l,y_l\right)$:
		$g_{T, \mathit{supervised} }^{\left(t\right)}={\left.{\nabla }_{{\theta }_T}CE\left(y_l,CL{F}_T\left(em{b}_l^T;{\theta }_T\right)\right)\right|}_{{\theta }_T={\theta }_T^{\left(t\right)}}$
		Compute feedback factor $h$ from student:
		$h=CE\left(y_l,S\left(x_l;{\theta }_S^{(t+1)}\right)\right)-CE\left(y_l,S\left(x_l;{\theta }_S^{\left(t\right)}\right)\right)$
		Compute MPL loss from unlabeled data $x_l$:
		$g_T^{\left(t\right)}={\left.h\cdot {\nabla }_{{\theta }_T}CE\left({\hat{y}}_u,CL{F}_T\left(em{b}_u^T;{\theta }_T\right)\right)\right|}_{{\theta }_T={\theta }_T^{\left(t\right)}}$
		Update the teacher classifier $CL{F}_T$:
		${\theta }_T^{(t+1)}={\theta }_T^{\left(t\right)}-\eta s\cdot \left(g_T^{\left(t\right)}+g_{T, \mathit{supervised} }^{\left(t\right)}\right)$
	end for return ${\theta }_{S,ENC}^{\left(N\right)},{\theta }_S^{\left(N\right)}$      $⊳$Only the student encoder and classifier are returned for evaluations.

4. Experiment

In this chapter, we first describe the experimental setup. Subsequently, we present the experimental results for Figure 2a,b in Section 4.2 and Section 4.3, respectively. Our main results consist of comparisons with the pretrained model and SupCon model in Section 4.4 and comparisons with state-of-the-art models in Section 4.5.

4.1. Experiment Setup

The experiments are conducted using NVIDIA Tesla V100 32 and NVIDIA RTX-3090 (NVIDIA, Santa Clara, CA, USA) with ubuntu 20.04 environment for reproducibility and stability. The single-domain experiment and the multi-domain experiment are existing outputs of the Meta Learning-based Deepfake Detection Project [24]. The pretrained model, Meta Pseudo Labels model (MPL model), SupCon model, and SupCon-MPL model are experimented for their training performance under the same conditions and hyperparameters. The training dataset uses the videos of the deepfakes (DF), Face2Face (F2F), FaceSwap (FS), NeuralTextures (NT), and real videos in FaceForensics++ [39], with DFDC [40], and Celeb-DF [41]. In scenario evaluation, deepfake videos of first-generation’s unknown domain are NeuralTextures(NT) [39] with real videos, and for post-generation’s unknown domain, we selected StyleGAN [8] images with CelebA [48] videos. We used MTCNN [49] to extract face images frame by frame of each video.

In the single-domain experiment, 260,000 real and 340,000 fake data from each domain in the FaceForensics++ [39] are used. During training, the amount of validation data used is 20% of the training data, and the evaluation dataset uses 150,000 per each data domain. In the multi-domain experiment, 200,000 labeled data are randomly extracted from four domains, and 180,000 unlabeled data are extracted from a single domain for use.

The generational deepfake scenario trains using 170,000 each of the first-generation known domain’s FaceForensics++ (DF, F2F, FS, Real) [39], DFDC [40], Celeb-DF [41] data, and then evaluates using 51,200 each of first- and second-generation unknown domain data. In the backbone model in scenario evaluation, we used ResNet50 [50] due to lack of computational resources.

The hyperparameters used in the experiment are a learning rate of 10⁻⁴, an image size of 64, and a batch size of 512. In the MPL, SupCon-MPL models, the batch sizes of labeled and unlabeled images are 64 and 448, respectively. Finally, the threshold is set at 0.95. The training models used were ResNet50 [50], ResNet101 [50], ResNext50 [51], EfficientNet-b5 [21], and WideResNet50 [52].

Experiment data and evaluation data use a mix of fake and real data. In the experiment in Section 4.2, video data from one domain is used as labeled and unlabeled data, and in the experiment in Section 4.3, videos from multiple domains are used as labeled data, and one domain is used as unlabeled data.

4.2. Single-Domain Experiment

In this section, single-domain experiments evaluate the detection performance for a single deepfake generation model, while, in Section 4.3, the performance on new deepfakes is evaluated through training on various deepfake generation models. Section 4.4 and Section 4.5 assess the performance of deepfake detection across diverse deepfake datasets and scenario-based deepfake detection, respectively.

In the experiment using only one domain, as shown in

Table 1. The performance of pretrained and MPL model on the known domain (EfficientNetb5 [21]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
EfficientNetb5 [21]	DF	DF	89.35	89.35	89.19	90.08	90.08	90.36
	F2F	F2F	77.21	77.21	77.61	80.35	80.35	79.45
	FS	FS	84.52	84.31	83.20	87.90	87.43	85.98
	NT	NT	64.13	63.33	58.18	74.79	74.47	71.36

, the performance of the known domain increased in most of the situation. Furthermore, it was confirmed that the performance in unknown domains also increased in most of the situation. Based on EfficientNet-b5 [21] in Table 1, ACC and AUC in K improved by an average of 4.47% and 4.53%, respectively, and ACC in U improved by an average of 0.20%, but AUC decreased by an average of 0.13%. However, overall, out of a total of 64 ACC and AUC validations recorded in Table 1,

Table 2. The performance of pretrained and MPL model on the unknown domain (EfficientNetb5 [21]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
EfficientNetb5 [21]	DF	F2F	51.59	51.60	22.09	51.70	51.70	23.01
		FS	55.97	52.06	23.23	56.45	52.65	25.01
		NT	55.37	51.26	20.06	55.41	51.37	21.28
	F2F	DF	57.56	57.55	47.78	53.87	53.84	32.11
		FS	56.47	54.05	39.66	55.48	51.92	26.57
		NT	54.63	51.91	33.96	54.76	50.99	23.26
	FS	DF	57.25	57.22	39.84	57.61	57.88	36.54
		F2F	51.76	51.77	25.78	51.98	51.99	20.16
		NT	53.79	49.89	20.80	54.32	49.80	12.35
	NT	DF	58.72	58.71	52.73	61.03	61.00	52.78
		F2F	54.52	54.53	45.23	55.88	55.89	43.68
		FS	51.65	49.42	34.03	53.29	49.34	27.25

,

Table 3. The performance of pretrained and MPL model on the known domain (ResNet50 [50]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNet50 [50]	DF	DF	91.16	91.16	90.91	91.29	91.30	90.97
	F2F	F2F	82.47	82.48	81.84	83.98	83.97	82.75
	FS	FS	87.55	87.21	86.01	88.31	88.09	87.06
	NT	NT	74.96	74.28	71.20	76.22	75.84	73.15

,

Table 4. The performance of pretrained and MPL model on the unknown domain (ResNet50 [50]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNet50 [50]	DF	F2F	52.91	52.89	22.41	51.92	51.86	20.36
		FS	56.80	52.68	21.41	57.02	53.00	22.63
		NT	55.91	51.77	18.54	55.45	51.33	18.53
	F2F	DF	53.94	53.90	31.72	53.79	53.75	28.28
		FS	54.62	50.97	24.47	55.32	51.35	21.84
		NT	54.76	51.20	24.07	54.96	51.04	21.00
	FS	DF	56.34	56.30	33.89	59.57	59.53	41.50
		F2F	51.16	51.10	20.08	51.57	51.52	21.01
		NT	53.67	49.44	14.32	53.81	49.61	15.49
	NT	DF	60.45	60.43	49.93	60.17	60.15	50.85
		F2F	54.74	54.70	38.88	53.56	53.51	36.77
		FS	51.59	48.15	22.30	51.84	48.51	25.20

,

Table 5. The performance of pretrained and MPL model on the known domain (ResNet101 [50]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNet101 [50]	DF	DF	91.16	91.16	90.84	91.13	91.13	90.85
	F2F	F2F	81.41	81.41	80.27	83.50	83.49	82.73
	FS	FS	87.59	87.37	86.13	87.75	87.66	86.39
	NT	NT	74.17	74.56	73.16	76.03	75.53	73.22

,

Table 6. The performance of pretrained and MPL model on the unknown domain (ResNet101 [50]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNet101 [50]	DF	F2F	52.69	52.63	21.22	51.26	51.20	18.01
		FS	56.90	52.73	20.42	56.02	51.84	18.71
		NT	55.93	51.74	18.33	54.93	50.73	16.53
	F2F	DF	53.63	53.59	30.44	53.50	53.46	27.42
		FS	54.33	50.08	22.82	54.86	50.85	19.67
		NT	54.62	51.01	23.30	55.17	51.28	20.98
	FS	DF	57.04	57.70	37.00	59.55	59.51	42.55
		F2F	51.43	51.37	20.63	51.84	51.79	24.60
		NT	53.66	49.55	15.04	53.61	49.59	17.59
	NT	DF	62.66	62.65	59.08	60.16	60.14	49.99
		F2F	52.69	52.63	47.55	51.26	51.20	37.97
		FS	56.90	52.73	29.98	56.02	51.84	21.29

,

Table 7. The performance of pretrained and MPL model on the known domain (ResNext50 [51]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Sore
ResNext50 [51]	DF	DF	90.29	90.29	90.01	91.14	91.14	91.02
	F2F	F2F	81.19	81.17	80.07	82.87	82.85	82.11
	FS	FS	87.04	86.77	85.44	87.36	87.36	86.36
	NT	NT	74.26	74.43	73.21	76.32	75.78	73.30

and

Table 8. The performance of pretrained and MPL model on the unknown domain (ResNext50 [51]).

Baseline Model	Train Dataset	Test Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNext50 [51]	DF	F2F	51.83	51.73	20.17	51.66	51.56	18.92
		FS	56.57	52.53	21.48	57.24	53.21	23.08
		NT	55.25	51.57	18.50	54.72	50.50	15.47
	F2F	DF	54.57	54.58	31.56	54.46	54.47	31.25
		FS	54.31	50.38	18.21	55.28	51.45	22.28
		NT	54.67	50.89	21.12	54.80	51.02	21.58
	FS	DF	59.23	59.23	41.25	60.33	60.33	46.22
		F2F	52.02	51.93	23.43	52.98	52.89	27.91
		NT	53.31	49.22	15.05	53.41	49.55	17.30
	NT	DF	60.58	60.58	43.46	58.50	58.51	44.82
		F2F	54.16	54.11	30.69	52.46	52.38	31.00
		FS	49.17	46.78	19.36	51.26	47.80	21.71

, 44 and 41 cases improved, respectively. Through this, it was confirmed that when using the same data, the performance of the MPL model is higher than the pretrained model.

4.3. Multi-Domain Experiment

In the multi-domain experiment, the combination of data is configured considering the actual deepfake situation. The situation is assumed to have $\mathit{K}$ deepfake data from multiple domains and $D_{UL}$ data from $\mathit{U}$. Afterwards, the evaluation is conducted through the $\mathit{U}$ data used as $D_{UL}$. Therefore, MPL trains with deepfake videos from multiple domains, and, after training, MPL experiments the ability to detect unlabeled data using unlabeled data. As a result of the experiment, ACC and AUC increased by an average of 1.59% and 1.26% in two models in

Table 9. The performance of pretrained and MPL model on the targeted unknown domain (ResNext50 [51]).

Baseline Model	Train Dataset	Unlabeled Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNext50 [51]	F2F, FS, NT	DF	63.33	63.32	70.02	66.23	66.22	70.35
	DF, FS, NT	F2F	57.35	57.39	60.69	58.04	58.08	61.64
	DF, F2F, NT	FS	50.69	50.53	46.69	51.81	51.54	47.22
	DF, F2F, FS	NT	53.63	53.05	47.65	53.89	53.06	46.28

and

Table 10. The performance of pretrained and MPL model on the targeted unknown domain (WideResNet50 [52]).

Baseline Model	Train Dataset	Unlabeled Dataset	Pretrained Model			MPL Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score
WideResNet50 [52]	F2F, FS, NT	DF	63.17	63.14	69.19	66.03	66.00	70.15
	DF, FS, NT	F2F	56.86	56.87	60.76	57.60	57.60	58.91
	DF, F2F, NT	FS	48.37	47.74	39.32	52.86	51.13	41.60
	DF, F2F, FS	NT	54.22	53.43	47.37	53.92	51.96	39.87

. Through this, it was confirmed that the MPL model improved the deepfake detection ability of the unknown model by learning with unlabeled data.

4.4. SupCon-MPL Experiment

The experiment uses the labeled data identically to the multi-domain experiment and evaluates according to each combination of known domain and unknown domain. At this time, Celeb-DF [41] is used as unlabeled data in the training of the SupCon-MPL model. As a result of the experiment, shown in

Table 11. The performance of SupCon-MPL compared with the baseline model and SupCon model [35].

Baseline Model	Train Dataset	Unlabeled Dataset	Pretrained Model			SupCon Model [35]			SupCon-MPL(Ours) Model
			ACC	AUC	F1 Score	ACC	AUC	F1 Score	ACC	AUC	F1 Score
ResNet50 [50]	FF (without DF)	DF (unknown)	64.24	64.27	65.49	62.88	62.84	58.55	64.60	64.55	61.60
		F2F + FS + NT (known)	70.56	71.36	83.36	75.44	75.52	83.01	75.84	76.00	83.50
	FF (without F2F)	F2F (unknown)	55.76	55.64	40.63	56.61	56.64	49.47	58.74	58.77	51.52
		DF + FS + NT (known)	77.26	76.89	81.66	75.54	75.54	84.13	78.11	78.28	85.84
	FF (without FS)	FS (unknown)	54.47	52.07	79.18	55.75	53.68	77.20	55.72	53.41	79.29
		DF + F2F + NT (known)	75.99	75.87	81.66	76.02	75.88	84.84	77.22	77.12	85.84
	FF (without NT)	NT (unknown)	56.71	54.95	62.66	56.58	54.31	66.27	56.76	54.02	69.43
		DF + F2F + FS (known)	77.39	77.41	81.66	79.09	79.26	84.13	81.22	81.32	85.84

, when evaluating FS data as an unknown domain compared with the SupCon model [35], ACC and AUC decreased, but in other validations, the performance of the SupCon-MPL model was similar or higher than the performance of the two models being compared. This shows that the SupCon-MPL model has been trained as a generalized detection model compared with the existing deepfake detection model.

4.5. Deepfake Scenario Experiment

In this section, we construct a training scenario for a deepfake detection model in the real world and train the model. The scenario involves training a deepfake model with first-generation deepfake data, then experimenting with the detection of first-generation deepfakes (NT) that were not participated while training, and post-generation (second-generation) deepfakes (StyleGAN) that are newly developed and unknown.

Table 12. The performance of SupCon-MPL compared with other deepfake detection methods.

Model	Scenario Deepfakes (Known)	Current-Generation Deepfakes (Unknown)	Post-Generation Deepfakes (Unknown)
Tar [34]	52.40	44.62	49.96
DDT [31]	80.41	44.62	49.49
MPL [24]	79.82	56.53	43.16
SupCon [35]	79.01	56.66	47.77
SupCon-MPL (ours)	81.40	57.85	51.90

shows the training results of various models according to the scenario. As a result of the experiment, among various models, the SupCon-MPL model achieved highest performance in all scenario evaluations compared with other models.

4.6. Limitations

The main goal of the SupCon-MPL is to enhance the deepfake detection performance in unknown domains using meta-learning. Therefore, in this paper, we conducted experiments by reconfiguring a limited deepfake dataset into scenarios.

The main limitation is related to computing resources. As the training in Section 4.3. and Section 4.4. was conducted using NVIDIA RTX-3090, only ResNet50 [50] could be used as the backbone model in SupCon-MPL, which uses two models. The ResNet50 [50] model currently stands out among various image classification models for its stability and consistently decent performance scores. Additionally, within the given resources, it can be utilized for training the SupCon-MPL model. Subsequent experiments are needed with various backbone models and larger image sizes through more computing resources.

The next limitation is that we could not find a verified deepfake dataset for the third generation and higher. Further experiments are needed through the corresponding dataset in the future.

5. Conclusions

With the development of various deepfake generative models, it has become important to develop a generalized deepfake detection model that guarantees the detection performance of unknown domain deepfakes, not just the data of the domain used for training. The proposed SupCon-MPL allows for training from unlabeled data and performs contrastive learning to enhance the detection performance of unknown deepfakes. This enables contrastive learning with a diverse range of data, and, during training, the teacher model and the student model infer information and provide feedback to each other, facilitating the student model to surpass the performance of the teacher model.

Our model improved all detection performances over other generalized deepfake detection models’ known and unknown domains in deepfake scenario evaluation. Indeed, one of the significant features of SupCon-MPL is its ability to train models using a large amount of unlabeled data. This provides a method to enhance the model’s performance utilizing the countless images and videos available on the internet and so on. Through this, our model enables the training of a generalized deepfake detection model and provides robust detection capability for real-world deepfakes. SupCon-MPL can contribute to the detection of newly developed deepfake generative models as they become increasingly difficult to distinguish from real images, especially with the development of various deepfake generative models.

Future research will focus on improving the detection performance of higher generation deepfake images/videos using these methods. Additionally, studies on reducing the training cost of SupCon-MPL will be conducted. The goal is to develop a more efficient and economical deepfake detection model through these efforts.