Developer slvDev has published an open-source project that runs a 28.9-million-parameter language model directly on an ESP32-S3, a microcontroller that costs roughly $8. The model runs entirely on the chip itself, with no data sent to a server, generating text at roughly 9 tokens per second and outputting it to a small connected display. The achievement stands out because of how little working memory the ESP32-S3 has to work with, just 512KB of SRAM, and the previous largest language model run on comparable hardware topped out at around 260,000 parameters, making this roughly 100 times larger.
The workaround borrows a technique from Google's Gemma models called Per-Layer Embeddings. Most of a language model's parameters sit in an embedding table that the model only reads from rather than computes on. slvDev keeps that 25-million-parameter table in the chip's slower flash storage, pulling only about six rows, roughly 450 bytes, per token, while the smaller compute-heavy portion of the model stays in fast SRAM. The result is a large model that barely touches the chip's limited RAM, since most of it is never fully loaded at once. In terms of specs, the quantized model weighs in at 14.9MB at 4-bit precision, running on hardware with 8MB of PSRAM and 16MB of flash alongside the 512KB of SRAM.
The model isn't general-purpose. It was trained on the TinyStories dataset and produces short, mostly coherent stories, but it cannot answer questions, follow instructions, write code, or recall facts. The point of the project, per the author, is demonstrating the memory architecture rather than showcasing model capability. The firmware, wiring instructions, training code, and full benchmark writeup are available in the project's GitHub repository, along with the commit history documenting a parameter-counting bug the author found and corrected along the way.
