BIROn - Birkbeck Institutional Research Online

    Spatiotemporal attacks for embodied agents

    Liu, A. and Huang, T. and Liu, X. and Xu, Y. and Ma, Y. and Chen, X. and Maybank, Stephen J. and Tao, D. (2020) Spatiotemporal attacks for embodied agents. Lecture Notes in Computer Science 12362 , pp. 122-138. ISSN 0302-9743.

    ECCV-AL-20200710.pdf - Author's Accepted Manuscript

    Download (2MB) | Preview


    Adversarial attacks are valuable for providing insights into the blind spots of deep learning models and help improve their robustness. Existing work on adversarial attacks have mainly focused on static scenes; however, it remains unclear whether such attacks are effective against embodied agents, which could navigate and interact with a dynamic environment. In this work, we take the first step to study adversarial attacks for embodied agents. In particular, we generate spatiotemporal perturbations to form 3D adversarial examples, which exploit the interaction history in both the temporal and spatial dimensions. Regarding the temporal dimension, since agents make predictions based on historical observations, we develop a trajectory attention model to explore scene view contributions, which further help localize 3D objects appeared with the highest stimuli. By conciliating with clues from the temoral dimension, along the spatial dimension, we adversarially perturb the physical properties (e.g., texture and 3D shape) of the contextual objects that appeared in the most important scene views. Extensive experments on the EQA-v1 dataset for several emboded tasks in both the white-box and the black-box settings have been conducted, which demonstrate that our perturbations have strong attack and generalization abilities.


    Item Type: Article
    Additional Information: Computer Vision – ECCV 2020. 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVII. Also part of the Image Processing, Computer Vision, Pattern Recognition, and Graphics book sub series (LNIP, volume 12362).
    Keyword(s) / Subject(s): Embodied Agents, Spatiotemporal Perturbations, 3D Adversarial Examples
    School: Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences
    Depositing User: Stephen Maybank
    Date Deposited: 02 Dec 2020 17:40
    Last Modified: 09 Aug 2023 12:48


    Activity Overview
    6 month trend
    6 month trend

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item