BIROn - Birkbeck Institutional Research Online

    Tangent Fisher vector on matrix manifolds for action recognition

    Luo, G. and Wei, J. and Hu, W. and Maybank, Stephen J. (2019) Tangent Fisher vector on matrix manifolds for action recognition. IEEE Transactions on Image Processing 29 , pp. 3052-3064. ISSN 1057-7149.

    TangentFisherVector.pdf - Author's Accepted Manuscript

    Download (2MB) | Preview


    In this paper, we address the problem of representing and recognizing human actions from videos on matrix manifolds. For this purpose, we propose a new vector representation method, named tangent Fisher vector, to describe video sequences in the Fisher kernel frame work. We first extract dense curved spatio-temporal cuboids from each video sequence. Compared with the traditional 'straight cuboids', the dense curved spatio-temporal cuboids contain much more local motion information. Each cuboid is then described using a linear dynamical system (LDS) to simultaneously capture the local appearance and dynamics. Furthermore, a simple yet efficient algorithm is proposed to learn the LDS parameters and approximate the observability matrix at the same time. Each video sequence is thus represented by a set of LDSs. Considering that each LDS can be viewed as a point in a Grassmann manifold, we propose to learn an intrinsic GMM on the manifold to cluster the LDS points. Finally, a tangent Fisher vector is computed by first accumulating all the tangent vectors in each Gaussian component, and then concatenating the normalized results across all the Gaussian components. A kernel is defined to measure the similarity between tangent Fisher vectors for classification and recognition of a video sequence. This aproach is evaluated on the state-of-the-art human action benchmark datasets. The recognition performance is competitive when compared with current state-of-the-art results.


    Item Type: Article
    Additional Information: (c) 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.
    Keyword(s) / Subject(s): Action recognition, Fisher vector, Grassmann manifold, Hankel matrix, matrix manifold
    School: School of Business, Economics & Informatics > Computer Science and Information Systems
    Depositing User: Stephen Maybank
    Date Deposited: 25 Nov 2019 15:14
    Last Modified: 21 Sep 2022 18:23


    Activity Overview
    6 month trend
    6 month trend

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item