Comparison of different learning rate (step size) on Logistic regression using FR conjugate gradient optimizer

Abstract

Conjugate gradient algorithm is one of the effective optimization algorithms used in solving logistic regression problems. This paper is focused on comparing some existing learning rate methods to reduce the objective function value of the logistic regression model with a limited number of iterations and reduced processing time. Fletcher-Reeves (FR) conjugate gradient method was run in python program using admission and iris flowers dataset to examine the performance of each learning rate. The numerical results of each step size were compared. The result shows that Armijo step size performs better in terms of number of iterations and processing time with good model accuracy.