Selective particle attention: rapidly and flexibly selecting features for deep reinforcement learning
Blakeman, Sam and Mareschal, Denis (2022) Selective particle attention: rapidly and flexibly selecting features for deep reinforcement learning. Neural Networks 150 , pp. 408-421. ISSN 0893-6080.
|
Text
47766.pdf - Author's Accepted Manuscript Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (1MB) | Preview |
Abstract
Deep Reinforcement Learning (RL) is often criticized for being data inefficient and in exible to changes in task structure. Part of the reason for these issues is that Deep RL typically learns end-to-end using backpropagation, which results in task-specifc representations. One approach for circumventing these problems is to apply Deep RL to existing representations that have been learned in a more task-agnostic fashion. However, this only partially solves the problem as the Deep RL algorithm learns a function of all pre-existing representations and is therefore still susceptible to data inefficiency and a lack of exibility. Biological agents appear to solve this problem by forming internal representations over many tasks and only selecting a subset of these features for decision-making based on the task at hand; a process commonly referred to as selective attention. We take inspiration from selective attention in biological agents and propose a novel algorithm called Selective Particle Attention (SPA), which selects subsets of existing representations for Deep RL. Crucially, these subsets are not learned through backpropagation, which is slow and prone to overfitting, but instead via a particle filter that rapidly and exibly identifies key subsets of features using only reward feedback. We evaluate SPA on two tasks that involve raw pixel input and dynamic changes to the task structure, and show that it greatly increases the efficiency and exibility of downstream Deep RL algorithms.
Metadata
| Item Type: | Article |
|---|---|
| Keyword(s) / Subject(s): | Selective Attention, Visual Features, Reinforcement Learning, Particle Filter, Neural Networks |
| School: | Birkbeck Faculties and Schools > Faculty of Science > School of Psychological Sciences |
| Research Centres and Institutes: | Brain and Cognitive Development, Centre for (CBCD) |
| Depositing User: | Administrator |
| Date Deposited: | 14 Mar 2022 16:42 |
| Last Modified: | 02 Aug 2023 18:15 |
| URI: | https://eprints.bbk.ac.uk/id/eprint/47766 |
Statistics
Additional statistics are available via IRStats2.
Archive Staff Only (login required)
 |
Edit/View Item |