LLMs Without Deep Neural Networks? New Architecture Challenges Conventional Wisdom

SURPRISING CLAIM: LLMs can work effectively without deep neural networks.

Vincent Granville turned conventional wisdom on its head by demonstrating a working LLM architecture that does not use deep neural networks at all.

This changes everything we thought we knew about large language models.

Here's what they discovered: 🧵

---

2/ Everyone thought: Deep neural networks are essential for LLMs.

Multi-layer transformations, activation functions, back-propagation - these were considered fundamental.

But the data shows: You can achieve similar results with a completely different approach.

3/ Why we were wrong:

We assumed that depth was necessary for handling long-range dependencies and complex reasoning. The paper shows that with the right architecture and training approach, depth can be replaced with alternative mechanisms.

4/ What they built:

Granville built his LLM on an RBF (Radial Basis Function) network — the same machinery independently explored by Chinese researchers as a DNN substitute. The key innovation: his model finds the global optimum of the loss function in closed form, in a single iteration, eliminating gradient-descent training entirely.

5/ The results:

• Performance comparable to traditional transformers on many tasks
• Significantly faster training times
• Reduced computational requirements
• Better interpretability in some cases

6/ Why this matters:

If you're building AI systems, you should consider alternative architectures that do not rely on deep learning. This could lead to more efficient, explainable, and maintainable AI systems.

7/ The bigger picture:

This suggests that the current deep learning orthodoxy might be limiting innovation. There may be entirely different ways to build intelligent language systems that we have not explored because we are so focused on neural networks.

---

Paper: https://arxiv.org/abs/2605.30385

Follow @jaritz for more research → builder insights

---

Get the next paper breakdown in your inbox → Subscribe at patrick.technology