Hyperspherical Forward-Forward with Prototypical Representations

Abstract

The Forward-Forward (FF) algorithm presents a compelling, bio-inspired alternative to backpropagation. However, while efficient in training, it has a computationally prohibitive inference process that requires a separate forward pass for every class that is evaluated. In this work, we introduce the Hyperspherical Forward-Forward (HFF), a novel reformulation that resolves this critical bottleneck. Our core innovation is to reframe the local objective of each layer from a binary goodness-of-fit task to a direct multi-class classification problem within a hyperspherical feature space. We achieve this by learning a set of class-specific, unit-norm prototypes that act as geometric anchors and implicit negatives. This architectural innovation preserves the benefits of local training while enabling weight update and inference in a single forward pass, making it >40x faster than the original FF algorithm. Our method is simple to implement, scales effectively to modern convolutional architectures, and achieves superior accuracy on standard image classification benchmarks, closing the gap with backpropagation. Most notably, we are among the first greedy local-learning methods to report over 25% top-1 accuracy on ImageNet-1k, and 65.96% with transfer learning.

Citation

If you use this work, please cite our paper:

@misc{sarode2026hff,
  title = {Hyperspherical Forward-Forward with Prototypical Representations},
  author = {Shalini Sarode and Brian Moser and Joachim Folz and Federico Raue
            and Tobias Nauen and Stanislav Frolov and Andreas Dengel},
  year = {2026},
  eprint = {2605.00082},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
}