RBF Networks: Predicting by Similarity to Centers Found with k-Means | NTU Hsuan-Tien Lin Machine Learning Techniques Lecture 14
The result of the Gaussian SVM can be read as a linear vote over Gaussians centered at the support vectors; let both the centers and the votes be learned and you have the RBF network. When all N examples serve as centers, the optimal votes for regression are β = Z⁻¹y and the training error is exactly 0, so you regularize with ridge or by cutting the centers down to M ≪ N prototypes. The prototypes are found with k-means, which alternates two steps, assigning every example to its closest center and moving every center to the average of its cluster, until the clusters stop changing.