BIROn - Birkbeck Institutional Research Online

    Selective particle attention: rapidly and flexibly selecting features for deep reinforcement learning

    Blakeman, Sam and Mareschal, Denis (2022) Selective particle attention: rapidly and flexibly selecting features for deep reinforcement learning. Neural Networks , ISSN 0893-6080. (In Press)

    [img] Text
    47766.pdf - Author's Accepted Manuscript
    Restricted to Repository staff only
    Available under License Creative Commons Attribution Non-commercial No Derivatives.

    Download (1MB) | Request a copy

    Abstract

    Deep Reinforcement Learning (RL) is often criticized for being data inefficient and in exible to changes in task structure. Part of the reason for these issues is that Deep RL typically learns end-to-end using backpropagation, which results in task-specifc representations. One approach for circumventing these problems is to apply Deep RL to existing representations that have been learned in a more task-agnostic fashion. However, this only partially solves the problem as the Deep RL algorithm learns a function of all pre-existing representations and is therefore still susceptible to data inefficiency and a lack of exibility. Biological agents appear to solve this problem by forming internal representations over many tasks and only selecting a subset of these features for decision-making based on the task at hand; a process commonly referred to as selective attention. We take inspiration from selective attention in biological agents and propose a novel algorithm called Selective Particle Attention (SPA), which selects subsets of existing representations for Deep RL. Crucially, these subsets are not learned through backpropagation, which is slow and prone to overfitting, but instead via a particle filter that rapidly and exibly identifies key subsets of features using only reward feedback. We evaluate SPA on two tasks that involve raw pixel input and dynamic changes to the task structure, and show that it greatly increases the efficiency and exibility of downstream Deep RL algorithms.

    Metadata

    Item Type: Article
    Keyword(s) / Subject(s): Selective Attention, Visual Features, Reinforcement Learning, Particle Filter, Neural Networks
    School: School of Science > Psychological Sciences
    Research Centres and Institutes: Brain and Cognitive Development, Centre for (CBCD)
    Depositing User: Administrator
    Date Deposited: 14 Mar 2022 16:42
    Last Modified: 16 Mar 2022 05:40
    URI: https://eprints.bbk.ac.uk/id/eprint/47766

    Statistics

    Activity Overview
    6 month trend
    0Downloads
    6 month trend
    34Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item