Customised ensemble methodologies for deep learning: boosted residual networks and related approaches

Mosca, Alan and Magoulas, George D. (2018) Customised ensemble methodologies for deep learning: boosted residual networks and related approaches. Neural Computing and Applications 31 (6), pp. 1713-1731. ISSN 0941-0643.

Abstract

This paper introduces a family of new customised methodologies for ensembles, called Boosted Residual Networks (BRN), which builds a boosted ensemble of Residual Networks by growing the member network at each round of boosting. The proposed approach combines recent developements in Residual Networks - a method for creating very deep networks by including a shortcut layer between different groups of layers - with Deep Incremental Boosting, a methodology to train fast ensembles of networks of increasing depth through the use of boosting. Additionally, we explore a simpler variant of Boosted Residual Networks based on Bagging, called Bagged Residual Networks (BaRN). We then analyse how the recent developments in Ensemble distillation can improve our results.We demonstrate that the synergy of Residual Networks and Deep Incremental Boosting has better potential than simply boosting a Residual Network of fixed structure or using the equivalent Deep Incremental Boosting without the shortcut layers, by permitting the creation of models with better generalisation in significantly less time.

Metadata

Available Versions of this Item

• Customised ensemble methodologies for deep learning: boosted residual networks and related approaches. (deposited 04 Feb 2019 11:51) [Currently Displayed]

Statistics

DownloadsShow export options

Export as XMLJSONCSV

Activity Overview

6 month trend

321Downloads

6 month trend

287Hits

Additional statistics are available via IRStats2.

Archive Staff Only (login required)

Edit/View Item