A new behavior-based cost function for manifold construction

Abstract

In backpropagation (BP) neural network learning algorithm, the mean squared error (MSE) has been used as a cost function. The cost function measures the squared difference between the target and the output. The real difference may be either positive or negative depending on whether the target is above or below the actual output. Obviously, the generated manifold obtained by the cost function does not preserve the real topology of the target manifold at any location. These disadvantages cause the slow convergent rate. Therefore, the results show an ineffective forecasting. The objective of this research is to propose a new behavior-based cost function to capture the parallel behavior between the target and the output. The method is based on the dot product between the target and the output vectors. If the dot product is sufficiently close to zero, then the training can be terminated. Therefore, the number of steps needed to train the network can be minimized. The proposed cost function is tested with the benchmarked test sets. The performance is evaluated by measuring the values of cosine between the actual target and the output. The accuracy of the proposed methodology is over 90%, and it can reduce the amount of the learning process at least two times. The difference of the proposed method and the standard methodology is also checked by the t-test value.