BIROn - Birkbeck Institutional Research Online

    Customised ensemble methodologies for deep learning: boosted residual networks and related approaches

    Mosca, Alan and Magoulas, George (2018) Customised ensemble methodologies for deep learning: boosted residual networks and related approaches. Neural Computing and Applications 31 (6), pp. 1713-1731. ISSN 0941-0643.

    This is the latest version of this item.

    [img] Text
    NCA-Biron.pdf - Author's Accepted Manuscript
    Restricted to Repository staff only until 11 December 2019.

    Download (552kB)

    Abstract

    This paper introduces a family of new customised methodologies for ensembles, called Boosted Residual Networks (BRN), which builds a boosted ensemble of Residual Networks by growing the member network at each round of boosting. The proposed approach combines recent developements in Residual Networks - a method for creating very deep networks by including a shortcut layer between different groups of layers - with Deep Incremental Boosting, a methodology to train fast ensembles of networks of increasing depth through the use of boosting. Additionally, we explore a simpler variant of Boosted Residual Networks based on Bagging, called Bagged Residual Networks (BaRN). We then analyse how the recent developments in Ensemble distillation can improve our results.We demonstrate that the synergy of Residual Networks and Deep Incremental Boosting has better potential than simply boosting a Residual Network of fixed structure or using the equivalent Deep Incremental Boosting without the shortcut layers, by permitting the creation of models with better generalisation in significantly less time.

    Metadata

    Item Type: Article
    Additional Information: The final publication is available at Springer via the link above.
    School: Birkbeck Schools and Departments > School of Business, Economics & Informatics > Computer Science and Information Systems
    Research Centre: Birkbeck Knowledge Lab
    Depositing User: Prof George Magoulas
    Date Deposited: 04 Feb 2019 11:51
    Last Modified: 08 Aug 2019 18:25
    URI: http://eprints.bbk.ac.uk/id/eprint/25541

    Available Versions of this Item

    • Customised ensemble methodologies for deep learning: boosted residual networks and related approaches. (deposited 04 Feb 2019 11:51) [Currently Displayed]

    Statistics

    Downloads
    Activity Overview
    5Downloads
    50Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item