BIROn - Birkbeck Institutional Research Online

    Feedback graph convolutional network for skeleton-based action recognition

    Yang, H. and Yan, D. and Zhang, L. and Sun, Y. and Li, D. and Maybank, Stephen (2021) Feedback graph convolutional network for skeleton-based action recognition. IEEE Transactions on Image Processing , ISSN 1057-7149. (In Press)

    [img]
    Preview
    Text
    FeedBackGraph.pdf - Author's Accepted Manuscript

    Download (2MB) | Preview

    Abstract

    Skeleton-based action recognition has attracted considerable attention since the skeleton data is more robust to the dynamic circumstances and complicated backgrounds than other modalities. Recently, many researchers have used the Graph Convolutional Network (GCN) to model spatio-temporal features of skeleton sequences by an end-to-end optimization. However, conventional GCNs are feedforward networks for which it is imoossible for the shallower layers to access semantic information in the high-level layers. In this paper we propose a novel network, named Feedback Graph Convolutional Network (FGCN). This is the first work that introduces a feedback mechanism into GCNs for action recognition. Compared with conventional GCNs, FGCN has the following advantages: (1) A multi-staged temporal sampling strategy is designed to extract spatio-temporal features for action recognition in a coarse to fine process; (2) A Feedback Graph Convolutional Block (FGCB) is proposed to introduce dense feedback connections into the GCNs. It transmits the high level semantic features to the shallower layers and conveys temporal information stage by stage to model video level spatial-temporal features for action recognition; (3) The FGCN model provides predictions on-the-fly. In the early stages, its predictions are relatvely coarse. These coarse predictions are treated as priors to guide the feature learning in later stages, to obtain more accurate predictions. Extensive experiments on three datasets NTU-RGB+D, NTU-RGB+D120 and Northwestern-UCLA, demonstate that the proposed FGCN is effective for action recognition. It achieves the state-of-the-art performance on all three datasets.

    Metadata

    Item Type: Article
    Keyword(s) / Subject(s): Feedback Mechanism, Graph Convolutional Network, Skeleton, Action Recognition
    School: School of Business, Economics & Informatics > Computer Science and Information Systems
    Depositing User: Steve Maybank
    Date Deposited: 15 Nov 2021 11:43
    Last Modified: 15 Dec 2021 01:10
    URI: https://eprints.bbk.ac.uk/id/eprint/46540

    Statistics

    Downloads
    Activity Overview
    101Downloads
    21Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item