Improved Encoder-Decoder Temporal Action Detection Algorithm

Yue Wang; Hansong Su; Gaohua Liu

doi:10.3788/LOP202158.2020001

Journals >Laser & Optoelectronics Progress >Volume 58 >Issue 20 >Page 2020001 > Article

Laser & Optoelectronics Progress
Vol. 58, Issue 20, 2020001 (2021)

Improved Encoder-Decoder Temporal Action Detection Algorithm

Yue Wang, Hansong Su, and Gaohua Liu^*

Author Affiliations

School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China

show less

DOI: 10.3788/LOP202158.2020001 Cite this Article Set citation alerts

Yue Wang, Hansong Su, Gaohua Liu. Improved Encoder-Decoder Temporal Action Detection Algorithm[J]. Laser & Optoelectronics Progress, 2021, 58(20): 2020001 Copy Citation Text

show less

Fig. 1. Structure of the improved encoder-decoder temporal convolutional neural network

Download full size

Fig. 2. Structure of the residual module

Download full size

Fig. 3. Different feature fusion methods

Download full size

Fig. 4. Schematic diagram of traditional upsampling and improved upsampling. (a) Traditional upsampling; (b)improved upsampling

Download full size

Fig. 5. Detection example of MERL Shopping dataset

Download full size

Fig. 6. Detection example of GTEA dataset

Download full size

Block	Kernel size	Number of channels
Conv1	7×7	64
Conv2_x	$[\begin{array}{l} 1 \times 1 \\ 3 \times 3 \\ 1 \times 1 \end{array}]$ ×3	$[\begin{array}{l} 64 \\ 64 \\ 256 \end{array}]$ ×3
Conv3_x	$[\begin{array}{l} 1 \times 1 \\ 3 \times 3 \\ 1 \times 1 \end{array}]$ ×4	$[\begin{array}{l} 128 \\ 128 \\ 512 \end{array}]$ ×4
Conv4_x	$[\begin{array}{l} 1 \times 1 \\ 3 \times 3 \\ 1 \times 1 \end{array}]$ ×6	$[\begin{array}{l} 256 \\ 256 \\ 1024 \end{array}]$ ×6
Conv5_x	$[\begin{array}{l} 1 \times 1 \\ 3 \times 3 \\ 1 \times 1 \end{array}]$ ×3	$[\begin{array}{l} 512 \\ 512 \\ 2048 \end{array}]$ ×3

Table 1. Parameters of feature extraction network

Dataset	Action	Accuracy /%
MERL Shopping	Reach to shelf	77.8
	Retract from shelf	79.3
	Hand in shelf	81.6
	Inspect the product	80.4
	Inspect the shelf	81.2

Table 2. Recognition accuracy rate of each action

Dataset	VggNet16	ResNet50	ED-TCN	Improved ED-TCN	mAP /%
MERL Shopping	√		√		24.3
MERL Shopping		√	√		25.6
MERL Shopping		√		√	29.3
GTEA	√		√		25.8
GTEA		√	√		27.2
GTEA		√		√	30.2

Table 3. Effectiveness of various module on the algorithm

Dataset	ED-TCN	Improved ED-TCN	Seg-F1@10	Seg-F1@25	Seg-F1@50
MERL Shopping	√		86.7	85.1	72.9
MERL Shopping		√	89.2	87.4	74.8
GTEA	√		72.2	69.3	56.0
GTEA		√	76.8	71.9	58.5

Table 4. Seg-F1 of different algorithms on different datasets

Algorithm	Accuracy /%	mAP /%	Seg-F1@10	Seg-F1@25	Seg-F1@50
MSN Det	64.6	29.5	46.4	42.6	25.6
MSN Seg	76.3	24.2	80.0	78.3	65.4
Dilated TCN	76.4	26.3	79.9	78.0	67.5
ED-TCN	79.0	25.5	86.7	85.1	72.9
Improved ED-TCN	82.4	29.3	89.2	87.4	74.8

Table 5. Comparison of results of different algorithms on the MERL Shopping dataset

Yue Wang, Hansong Su, Gaohua Liu. Improved Encoder-Decoder Temporal Action Detection Algorithm[J]. Laser & Optoelectronics Progress, 2021, 58(20): 2020001

Download Citation

Set citation alerts for the article

Tools

Set citation alerts for the article

Save the article for my favorites

Paper Information