An open-source project has squeezed a 28.9-million-parameter language model onto a microcontroller that costs about $8, generating text entirely on the chip at roughly nine tokens per second with no network connection at all. The demonstration, published on GitHub and rapidly climbing the Hacker News front page, shows how far the frontier of small, local AI has moved in just a short time.
The work centers on the ESP32-S3, an inexpensive embedded chip popular with hobbyists and hardware makers. Running a 28.9-million-parameter model on such a small piece of silicon would have seemed implausible not long ago, and the project offers a vivid illustration of the broader push toward on-device AI that we track in our daily AI industry coverage. Where the cloud once felt mandatory for any real language model, increasingly capable systems are finding homes on the edge.
The numbers behind the build
The project lays out its specifications plainly. The model stores 28.9 million parameters, of which about 25 million live in a flash lookup table. It runs on an ESP32-S3 chip with 512KB of fast SRAM, 8MB of PSRAM, and 16MB of flash storage. In practice it reaches about 9.5 tokens per second end to end, or 9.7 tokens per second of pure compute, and the entire model occupies just 14.9 megabytes when stored at 4-bit precision.
Most striking is the comparison the author draws with prior work. The last language model known to run on a chip in this class held only 260,000 parameters. The new build holds roughly a hundred times more, a jump that comes not from a bigger chip but from rethinking where the model's data actually lives.
A memory trick borrowed from Gemma
The core problem is that a microcontroller has almost no fast memory. The ESP32-S3 offers just 512KB of SRAM, and conventionally the whole model would need to be reachable from there, which is why previous attempts were stuck at a few hundred thousand parameters.
The workaround rests on a simple observation. Most of a language model's parameters sit in an embedding table, which the model reads from rather than computes on. So instead of forcing that enormous 25-million-row table into scarce fast memory, the project leaves it in slow flash storage and pulls only the handful of rows each token actually needs, about 450 bytes at a time. The small portion of the model that performs the real computation stays in fast memory, while the bulk of the weights simply wait in flash, sampled a little at a time.
That technique is known as Per-Layer Embeddings, and it originated with Google's Gemma models, specifically Gemma 3n and Gemma 4, where it was designed for phones and GPUs. According to the project's author, nobody had previously tried the approach on a chip this small.
What it can do, and what it cannot
The model was trained on TinyStories, a dataset of simple children's stories, so its abilities are deliberately narrow. It writes short, mostly coherent stories, but it will not answer factual questions, follow complex instructions, write code, or reason about the world. The author is candid that this limitation stems from the small reasoning portion of the model, and that the memory trick does not change it.
What makes the project interesting is therefore not its output quality but its architecture. The achievement is fitting a model this large onto a chip this tiny, proving that the same techniques powering efficient models on phones and servers can be pushed all the way down to hardware that costs less than a sandwich.
Why on-device AI matters
The implications reach well beyond a neat technical demo. Because the model runs entirely on the chip, nothing is ever sent to a server. That means total privacy by construction, no data leaves the device, and there is no cloud bill to pay. For applications in remote areas, low-power devices, or settings where connectivity is unreliable, that combination is genuinely useful.
It also points to a future in which small, specialized models are distributed across billions of cheap embedded devices rather than concentrated in a handful of giant data centers. The same pressures driving interest in local AI on laptops and phones, namely lower latency, lower cost, and stronger privacy, apply even more forcefully at the microcontroller scale.
An open invitation to tinker
The project is released under the permissive MIT license, and the author has shared the full code on GitHub along with detailed documentation and benchmark results. That openness means other hardware developers can study the approach, adapt it to other chips, or push the parameter count even higher.
Released under the handle slvDev, the work resonated strongly with the developer community, gathering hundreds of upvotes on Hacker News and prompting discussion about where the technique might travel next. If the trend holds, the line between a "language model" and an ordinary piece of embedded software is about to get a great deal blurrier.
Stay Ahead of AI
From $8 chips to billion-dollar data centers, the question of where AI runs is becoming as important as what AI can do. The hardware layer is evolving faster than most people realize, and the projects that look like curiosities today often define the mainstream tomorrow. Read more AI news on AI Buzz Wire to follow every breakthrough in edge computing, model efficiency, and on-device intelligence.
Keep up with the AI hardware revolution — visit AI Buzz Wire
