Feedback graph convolutional network for skeleton-based action recognition

Yang, H. and Yan, D. and Zhang, L. and Sun, Y. and Li, D. and Maybank, Stephen (2021) Feedback graph convolutional network for skeleton-based action recognition. IEEE Transactions on Image Processing 31 , pp. 164-175. ISSN 1057-7149.

Abstract

Skeleton-based action recognition has attracted considerable attention since the skeleton data is more robust to the dynamic circumstances and complicated backgrounds than other modalities. Recently, many researchers have used the Graph Convolutional Network (GCN) to model spatio-temporal features of skeleton sequences by an end-to-end optimization. However, conventional GCNs are feedforward networks for which it is imoossible for the shallower layers to access semantic information in the high-level layers. In this paper we propose a novel network, named Feedback Graph Convolutional Network (FGCN). This is the first work that introduces a feedback mechanism into GCNs for action recognition. Compared with conventional GCNs, FGCN has the following advantages: (1) A multi-staged temporal sampling strategy is designed to extract spatio-temporal features for action recognition in a coarse to fine process; (2) A Feedback Graph Convolutional Block (FGCB) is proposed to introduce dense feedback connections into the GCNs. It transmits the high level semantic features to the shallower layers and conveys temporal information stage by stage to model video level spatial-temporal features for action recognition; (3) The FGCN model provides predictions on-the-fly. In the early stages, its predictions are relatvely coarse. These coarse predictions are treated as priors to guide the feature learning in later stages, to obtain more accurate predictions. Extensive experiments on three datasets NTU-RGB+D, NTU-RGB+D120 and Northwestern-UCLA, demonstate that the proposed FGCN is effective for action recognition. It achieves the state-of-the-art performance on all three datasets.

Metadata

Statistics

DownloadsShow export options

Export as XMLJSONCSV

Activity Overview

6 month trend

329Downloads

6 month trend

125Hits

Additional statistics are available via IRStats2.

Archive Staff Only (login required)

Edit/View Item