Synthetic Error Signal Propagation in Multi-Layer Perceptron Systems

Keywords:
Multi-Layer Perceptron, Backpropagation, Synaptic Parameter Training, Deep Learning, Artificial Intelligence

Abstract:
Today's artificial intelligence (AI) systems usually take the form of the artificial neural networks (ANN). One of the earliest ANN structures is the well-known multilayer perceptron. Determination of the values of the parameters of the multilayer perceptron is often performed by the error gradient descent method, called the gradient backpropagation method. In this method, the gradient of the criterion function, usually the sum of squares of output errors, is computed layer by layer from the output towards the input. In the classical form of this method, the gradient values for the parameters of the hidden and the input layers depend on the previously computed gradient values. This may lead to either the gradient values of deeper layers decreasing towards zero (the gradient collapse), or growing to very large values (the gradient explosion). In either case, the parameter optimisation becomes impossible. In this work, this problem has been addressed by decoupling the error gradients with respect of the parameters in one layer from the gradient values in other layers. This has been achieved by estimating the output error for each layer, here called the layer synthetic error, and then computing the corresponding gradient values for each layer separately, thus decoupling the optimisation of the parameters of one layer from that of another layer. The synthetic error method has been compared to the conventional backpropagation method in an experiment. The goal of the experiment has been the training of the ANN on real-world data, using both methods. The results obtained in the experiment are very encouraging, as the synthetic error method has shown some clear advantages.

CITATION:

IEEE format

Copy to clipboard

D. Đukić, “Synthetic Error Signal Propagation in Multi-Layer Perceptron Systems,” in Sinteza 2026 - International Scientific Conference on Information Technology, Computer Science, and Data Science, Belgrade, Singidunum University, Serbia, 2026, pp. 247-253. doi:10.15308/Sinteza-2026-247-253

APA format

Copy to clipboard

Đukić, D. (2026). Synthetic Error Signal Propagation in Multi-Layer Perceptron Systems. Paper presented at Sinteza 2026 - International Scientific Conference on Information Technology, Computer Science, and Data Science. doi:10.15308/Sinteza-2026-247-253

BibTeX format
Download

RefWorks Tagged format
Download