Yahya H. Zweiri Optimization of a ThreeTerm Backpropagation Algorithm Used for Neural Network Learning
1850 - 1855
2007
1
12
International Journal of Electrical, Computer, Energetic, Electronic and Communication Engineering http://waset.org/publications/14541
http://waset.org/publications/12
World Academy of Science, Engineering and Technology
The backpropagation algorithm calculates the weight
changes of an artificial neural network, and a twoterm algorithm
with a dynamically optimal learning rate and a momentum factor
is commonly used. Recently the addition of an extra term, called a
proportional factor (PF), to the twoterm BP algorithm was proposed.
The third term increases the speed of the BP algorithm. However,
the PF term also reduces the convergence of the BP algorithm, and
optimization approaches for evaluating the learning parameters are
required to facilitate the application of the three terms BP algorithm.
This paper considers the optimization of the new backpropagation
algorithm by using derivative information. A family of approaches
exploiting the derivatives with respect to the learning rate, momentum
factor and proportional factor is presented. These autonomously
compute the derivatives in the weight space, by using information
gathered from the forward and backward procedures. The threeterm
BP algorithm and the optimization approaches are evaluated using
the benchmark XOR problem.
International Science Index 12, 2007