Luo, G. and Wei, J. and Hu, W. and Maybank, Stephen J. (2019) Tangent Fisher vector on matrix manifolds for action recognition. IEEE Transactions on Image Processing 29 , pp. 3052-3064. ISSN 1057-7149.
|
Text
TangentFisherVector.pdf - Author's Accepted Manuscript Download (2MB) | Preview |
Abstract
In this paper, we address the problem of representing and recognizing human actions from videos on matrix manifolds. For this purpose, we propose a new vector representation method, named tangent Fisher vector, to describe video sequences in the Fisher kernel frame work. We first extract dense curved spatio-temporal cuboids from each video sequence. Compared with the traditional 'straight cuboids', the dense curved spatio-temporal cuboids contain much more local motion information. Each cuboid is then described using a linear dynamical system (LDS) to simultaneously capture the local appearance and dynamics. Furthermore, a simple yet efficient algorithm is proposed to learn the LDS parameters and approximate the observability matrix at the same time. Each video sequence is thus represented by a set of LDSs. Considering that each LDS can be viewed as a point in a Grassmann manifold, we propose to learn an intrinsic GMM on the manifold to cluster the LDS points. Finally, a tangent Fisher vector is computed by first accumulating all the tangent vectors in each Gaussian component, and then concatenating the normalized results across all the Gaussian components. A kernel is defined to measure the similarity between tangent Fisher vectors for classification and recognition of a video sequence. This aproach is evaluated on the state-of-the-art human action benchmark datasets. The recognition performance is competitive when compared with current state-of-the-art results.
Metadata
| Item Type: | Article |
|---|---|
| Additional Information: | (c) 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. |
| Keyword(s) / Subject(s): | Action recognition, Fisher vector, Grassmann manifold, Hankel matrix, matrix manifold |
| School: | Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences |
| Depositing User: | Stephen Maybank |
| Date Deposited: | 25 Nov 2019 15:14 |
| Last Modified: | 09 Aug 2023 12:47 |
| URI: | https://eprints.bbk.ac.uk/id/eprint/30050 |
Statistics
Additional statistics are available via IRStats2.
Archive Staff Only (login required)
