Fazaeli-Javan M, Monsefi R, Ghiasi-Shirazi K. Analyzing of Positive and Negative Prototypes based on the ±ED-WTA Method. Journal of Iranian Association of Electrical and Electronics Engineers 2025; 22 (2)
URL:
http://jiaeee.com/article-1-1662-en.html
Ferdowsi university of Mashhad
Abstract: (162 Views)
The recently introduced method, ±ED-WTA, for each output neuron of a class in the last layer of neural networks, obtain a pair of positive and negative prototypes. The functionality of a neuron is explained based on the difference in the Euclidean distance of a sample from these two prototypes. An astonishing point in this method is the great similarity of positive and negative prototypes for each neuron at the end of training. The authors of [2] have claimed that the reason for this extreme similarity is the formation of the negative prototype of a class with samples of other classes, which are very similar to the positive prototype of that class. In this paper, we show that it is not only the negative prototype that gets close to the positive prototype of a class but also a positive prototype is formed by samples that are far from the center of that class. The new finding about this great similarity shows that each neuron in the softmax layer, like SVMs, makes decisions based on the near-boundary samples. The theoretical analysis is examined in detail and experimental results on MNIST, FERET, and Fashion-MNIST show the correctness of the claims made.
Type of Article:
Research |
Subject:
Control Received: 2023/11/2 | Accepted: 2024/09/19