A Novel Strategy for Weight Initialization in Sigmoidal Feed-forward Artificial Neural Networks

Abstract

In this paper, a novel method of weight initialization is proposed. The proposed method of weight initialization distributes the initial weights and thresholds in such a manner that they lie in different regions of the activation function used at the hidden layer. The proposed method is compared with six other popular weight initialization methods on ten function approximation problems using the RPROP (Resilient Back-propagation) and Levenberg-Marquardt algorithms for training. Two types of activation functions viz. tan hyperbolic and logarithmic sigmoidal functions are used for analysis and comparison.