Exposing DeepFake Video Detection Based on Convolutional Long Short-Term Memory Network

Bowen Zheng; Huawei Xia; Ruidong Chen; Qiankun Han

doi:10.3788/LOP202158.2415002

Journals >Laser & Optoelectronics Progress >Volume 58 >Issue 24 >Page 2415002 > Article

Laser & Optoelectronics Progress
Vol. 58, Issue 24, 2415002 (2021)

Exposing DeepFake Video Detection Based on Convolutional Long Short-Term Memory Network

Bowen Zheng, Huawei Xia^*, Ruidong Chen^**, and Qiankun Han^***

Author Affiliations

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

show less

DOI: 10.3788/LOP202158.2415002 Cite this Article Set citation alerts

Bowen Zheng, Huawei Xia, Ruidong Chen, Qiankun Han. Exposing DeepFake Video Detection Based on Convolutional Long Short-Term Memory Network[J]. Laser & Optoelectronics Progress, 2021, 58(24): 2415002 Copy Citation Text

show less

Fig. 1. Framework of convolutional LSTM network

Download full size

Influence of number of frames on the classification accuracy on the FaceForensics++ dataset. (a) Gaussian blur frame number; (b) face data removal frame number; (c) total frame number

Fig. 2. Influence of number of frames on the classification accuracy on the FaceForensics++ dataset. (a) Gaussian blur frame number; (b) face data removal frame number; (c) total frame number

Download full size

Backbone network	Classification accuracy /%
Backbone network	LQ	HQ	RAW
AlexNet^[22]	96.89	88.64	91.97
VGG16^[11]	90.85	92.35	94.67
ResNet^[10]	93.42	95.68	96.43
EfficientNet(Ours)	96.51	97.89	99.57

Table 1. Comparison of different backbone networks on the FaceForensics++ dataset

Algorithm	Classification accuracy /%
Algorithm	LQ	HQ	RAW
CNN^[21]	90.00	91.45	93.40
SVM^[25]	70.10	73.64	75.43
RNN^[6]	93.46	95.04	95.98
GRU^[24]	94.48	96.18	97.54
LSTM^[14]	94.29	96.24	96.79
ConvLSTM^[19]	95.18	96.79	98.80
ConvLSTM(with attention)	96.51	97.89	99.57

Table 2. Comparison of different algorithms on the FaceForensics++ dataset

Model	Classification accuracy /%
Model	LQ	HQ	RAW
ConvLSTM	95.18	96.79	98.80
ConvLSTM+hard-attention	95.22	96.91	99.24
Self-attention	95.96	97.89	98.91
ConvLSTM+soft-attention	96.51	97.34	99.57

Table 3. Experimental results of different attention models on the FaceForensics++ dataset

Number of manipulatedframes	Classification accuracy /%
Number of manipulatedframes	LQ	HQ	RAW
2	71.32	78.41	82.17
3	85.66	87.63	89.68
4	88.97	90.05	92.53
5	92.65	94.32	97.43
6	75.74	79.57	82.64

Table 4. Classification accuracy of the variation of Gaussian blur frame numbers on the FaceForensics++ dataset

Number of manipulatedframes	Classification accuracy /%
Number of manipulatedframes	LQ		HQ		RAW
3		95.22		96.27		97.48
4		96.07		97.08		98.32
5		96.50		97.55		99.15
6		96.74		97.91		99.57
7		96.12		96.94		98.51
8		94.26		95.46		97.20
9		92.37		93.84		94.78

Table 5. Experimental results of the facial data removal on the FaceForensics++ dataset

Total framenumber	Classification accuracy /%
Total framenumber	LQ	HQ	RAW
5	93.43	95.06	95.95
10	96.51	97.32	98.22
15	96.70	98.49	99.91
20	96.74	98.08	99.05

Table 6. Classification accuracy of the variation of the total frame numbers on the FaceForensics++ dataset

Reference	Method	Classifier	Classification accuracy /%	Dataset
Ref.[1]	Mesoscopic features	CNN	83.2	F2F
Ref.[21]	Steganalysis features	CNN	91.0	F2F
			94.0	DF
			93.0	FS
			81.0	NT
Ref.[6]	Temporal features	RNN	94.3	F2F
Ref.[27]	Temporal features	Optical Flow	81.6	F2F
Ref.[28]	Deep learning features	3DCNN	95.1	DF
Ref.[28]	Deep learning features	3DCNN	92.3	FS
This work	Interframe features	ConvLSTM	96.5	F2F
			96.7	DF
			94.9	FS
			92.7	NT

Table 7. Comparison of classification experimental results of different algorithms on the FaceForensics++ dataset

Bowen Zheng, Huawei Xia, Ruidong Chen, Qiankun Han. Exposing DeepFake Video Detection Based on Convolutional Long Short-Term Memory Network[J]. Laser & Optoelectronics Progress, 2021, 58(24): 2415002

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information