LLMs Without DNNs: RBF Networks Challenge Transformer Orthodoxy
LLMs Without Deep Neural Networks?
Vincent Granville has demonstrated a working LLM architecture that uses no deep neural networks at all.
The assumption it challenges is a load-bearing one. Multi-layer transformations, activation functions and back-propagation are treated as fundamental to large language models, and depth is assumed necessary for long-range dependencies and complex reasoning.
What he built
The model runs on an RBF (Radial Basis Function) network, the same machinery Chinese researchers have independently explored as a DNN substitute. The key move is that it finds the global optimum of the loss function in closed form, in a single iteration, which removes gradient-descent training entirely.
Reported results
- Performance comparable to traditional transformers on many tasks
- Significantly faster training
- Lower computational requirements
- Better interpretability in some cases
Why it matters
If the results hold, depth is one available mechanism rather than a requirement, and a class of more efficient and more explainable systems becomes worth investigating. It also suggests the current deep learning orthodoxy is narrowing the search space rather than defining it.
Worth stating plainly: this is one author's architecture, benchmarked by that author. Treat the performance claims as unreplicated until someone independent runs them.
Stay in the loop
One dispatch per week — what I shipped, what broke, and what I learned from the field. No filler.
What should I write about?
Got a topic you'd like me to cover? I read every suggestion.