BIROn - Birkbeck Institutional Research Online

    Evolving stochastic learning algorithm based on Tsallis entropic index

    Anastasiadis, A.D. and Magoulas, George D. (2006) Evolving stochastic learning algorithm based on Tsallis entropic index. The European Physical Journal B - Condensed Matter and Complex Systems 50 (1-2), pp. 277-283. ISSN 1434-6028.

    [img]
    Preview
    Text
    502.pdf

    Download (263kB) | Preview

    Abstract

    In this paper, inspired from our previous algorithm, which was based on the theory of Tsallis statistical mechanics, we develop a new evolving stochastic learning algorithm for neural networks. The new algorithm combines deterministic and stochastic search steps by employing a different adaptive stepsize for each network weight, and applies a form of noise that is characterized by the nonextensive entropic index q, regulated by a weight decay term. The behavior of the learning algorithm can be made more stochastic or deterministic depending on the trade off between the temperature T and the q values. This is achieved by introducing a formula that defines a time-dependent relationship between these two important learning parameters. Our experimental study verifies that there are indeed improvements in the convergence speed of this new evolving stochastic learning algorithm, which makes learning faster than using the original Hybrid Learning Scheme (HLS). In addition, experiments are conducted to explore the influence of the entropic index q and temperature T on the convergence speed and stability of the proposed method.

    Metadata

    Item Type: Article
    Additional Information: Copyright © EDP Sciences, Società Italiana di Fisica, Springer-Verlag 2006. The original publication is available at www.springerlink.com
    School: Birkbeck Faculties and Schools > Faculty of Science > School of Computing and Mathematical Sciences
    Research Centres and Institutes: Birkbeck Knowledge Lab
    Depositing User: Sandra Plummer
    Date Deposited: 11 Jun 2007
    Last Modified: 09 Aug 2023 12:29
    URI: https://eprints.bbk.ac.uk/id/eprint/502

    Statistics

    Activity Overview
    6 month trend
    621Downloads
    6 month trend
    597Hits

    Additional statistics are available via IRStats2.

    Archive Staff Only (login required)

    Edit/View Item Edit/View Item