Faster Convergent Artificial Neural Networks
Proposed in this paper is a novel fast-convergence algorithm applied to neural networks (ANNs) with a learning rate based on the eigenvalues of the associated Hessian matrix of the input data. That is, the learning rate applied to the backpropagation algorithm changes dynamically with the input data used for training. The best choice of learning rate to converge to an accurate value quickly is derived. This newly proposed fast-convergence algorithm is applied to a traditional multilayer ANN architecture with feed-forward and backpropagation techniques. The proposed strategy is applied to various functions learned by the ANN through training. Learning curves obtained using calculated learning rates according to the novel method proposed are compared to learning curves utilizing an arbitrary learning rate to demonstrate the usefulness of this novel technique. This study shows that convergence to accurate values can be achieved much more quickly (a reduction in iterations by a factor of hundred) using the techniques proposed here. This approach is illustrated in this research work with derivations and pertinent examples to illustrate the method and learning curves obtained.
2. Margoulas, Androulakis, Vrahatis, 1999. Improving the convergence of the backpropagation algorithm using learning rate adaptation methods. Neural Computation (November 1999),1769-1796.
3. Haykin, S.,1994. Neural Networks: A Comprehensive Foundation. Macmillan Publishing Company, Englewood Cliffs, NJ.
4. â€œOptimal Weight and Learning Rates for Linear Networksâ€, Available online: URL: https://www.willamette.edu/~gorr/classes/cs449/LearningRates/LearningRates.html, 2017
Copyright (c) 2018 INTERNATIONAL JOURNAL OF COMPUTERS & TECHNOLOGY
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain the copyright of their manuscripts, and all Open Access articles are distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided that the original work is properly cited.
The use of general descriptive names, trade names, trademarks, and so forth in this publication, even if not specifically identified, does not imply that these names are not protected by the relevant laws and regulations. The submitting author is responsible for securing any permissions needed for the reuse of copyrighted materials included in the manuscript.
While the advice and information in this journal are believed to be true and accurate on the date of its going to press, neither the authors, the editors, nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.