BIROn - Birkbeck Institutional Research Online

    Solving the linear interval tolerance problem for weight initialization of neural networks

    Adam, S.P. and Karras, D.A. and Magoulas, George D. and Vrahatis, M.N. (2014) Solving the linear interval tolerance problem for weight initialization of neural networks. Neural Networks 54 , pp. 17-37. ISSN 0893-6080.

    [img]
    Preview
    Text
    LIT-Approach-web.pdf - Author's Accepted Manuscript

    Download (478kB) | Preview

    Abstract

    Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval Analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed Linear Interval Tolerance Approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods.

    Metadata

    Item Type: Article
    Keyword(s) / Subject(s): Neural networks, Weight initialization, Interval analysis, Linear interval tolerance problem
    School: Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences
    Research Centres and Institutes: Birkbeck Knowledge Lab
    Depositing User: George Magoulas
    Date Deposited: 22 Dec 2015 11:38
    Last Modified: 09 Aug 2023 12:37
    URI: https://eprints.bbk.ac.uk/id/eprint/13757

    Statistics

    Activity Overview
    6 month trend
    418Downloads
    6 month trend
    294Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item