How to Run Wikipedia Offline on ESP32

Submitted by Anand D on

These days, we are heavily dependent on cloud computing, high-speed internet and all. We rarely think about what happens if the internet goes down or what happens in areas where the network is not reachable or information access is very difficult. The main solution is having the data and its processing locally so that they are reachable anytime, even without internet. In this project, we will be building an Offline Wikipedia that runs completely on an ESP32-S3 Board so that we can access the data anytime, anywhere without an internet connection. This is all possible thanks to Kiwix, a non-profit organisation that is making free knowledge accessible everywhere. An ESP32 offline Wikipedia is a pocket-sized Wikipedia server built on an ESP32-S3 board with a microSD card. 

We will load the data onto an SD Card and use it in the onboard SD Card reader of the ESP32 module so that it itself becomes the database. Then we will flash the firmware that runs a web server on the ESP32 so that we can access the data in any browser without the internet. Running a web server on an ESP32 becomes very helpful, as we can visualise the data anytime, anywhere using any device. If you’d like to learn the basics of running a web server on an ESP32 Dev Module, we have a getting started tutorial that helps you build a simple ESP32 LED Webserver.

What You Will Learn in This ESP32 Offline Wikipedia Tutorial

  • Selection of the appropriate ZIM file (language, flavour, and size) for the microSD card.
  • Utilisation of the ESP32 offline Wikipedia repository on GitHub for the transformation of the ZIM file into data chunks easily readable by the ESP32.
  • Preparation and setting of the FAT32-formatted microSD card with the processed data.
  • Flashing the web server firmware using PlatformIO, along with checking SD card mounting.
  • Making connection to the ESP32 Wi-Fi point and search images-containing articles through any browser.
  • Overview of how this ESP32 version compares to setups with Raspberry Pi and smartphone Kiwix.

Hardware Required for the ESP32 Offline Wikipedia

Below is the hardware required to build this project.

Components for the ESP32 offline Wikipedia: an ESP32-S3 board with an onboard microSD slot, a microSD card and a USB data cable

First of all, we need an ESP32-S3 module with a microSD card slot. The memory card used here is a 4GB one for this demonstration. Memory card size depends on the file size that will be downloaded, as per the instructions provided below. A suitable memory card size is 16 GB or 32 GB as larger the file size, larger is the information that it holds. But still, one can even use a 2 GB memory card, provided the file size is within it. Then, we need a data cable to program the ESP32 Board.

Step-by-Step Guide to Build an Offline Wikipedia on ESP32

Step 1⇒ Download the Right Kiwix ZIM File
First of all, we need to go to the Kiwix ZIM Archive Page. There are several downloadable files listed. If we take a closer look, the letters “en, af, ru, fr…” in the file name suggest that the downloadable file is in a specific language, such as English, Afrikaans, Russian, French and many more. Below is a table that shows how to properly identify the right file for the right purpose. Make sure that the downloadable file size is within the capacity of the memory card that is going to be used. 

S NoTag                                                            Meaning
1simpleSimple English Wikipedia (easier vocabulary, fewer total articles, significantly smaller file sizes)
2allContains general Wikipedia articles across all topics rather than a specific subject. 
3maxiFull version including complete text and images
4nopicText-only version (no pictures), resulting in much smaller file sizes.  
mini: Lead/intro paragraphs of articles only
52026-xxThe year and month the snapshot was taken

Apart from Wikipedia, there are files specific to topics like physics, mathematics, football, chess, etc., which have rich information about the specific topics. Make sure to check the file size of the download as well. If we don't want images in the offline wiki, we can download the file that has “nopic” in the file name.

Topic-specific Kiwix archives let you build a focused offline reference, for example only physics or mathematics

The table below shows how languages are tagged in the file name. Identify the desired language and download the file. Listed below are just some samples. Actually, a wide range of languages are available on the website for download.

Language codes used in Kiwix Wikipedia ZIM file names for the ESP32 offline Wikipedia, such as en, fr, ru and af Language codes in Kiwix file names. Choose the code for the language you want to read offline

Step 2⇒ Clone the ESP32 Offline Wikipedia GitHub Repository
We will complete the entire tutorial in VS Code with the PlatformIO extension installed. To make everything easier, all the code and files have been uploaded to GitHub and can be accessed from the GitHub link provided below. After cloning the repo, we need to be in the following directory in the terminal for the following steps.

C:\Users\Pavilion\Downloads\Offline-Wikipedia-ESP32-master\Offline-Wikipedia-ESP32-master>

Step 3⇒ Preprocess the ZIM File with Images
Below is the command to be run to preprocess/re-compress the contents of the .zim file so that the ESP32 board can read the data efficiently. It will extract all articles, render all images/diagrams into .dat chunks, and generate the index files. The default thumbnail size is 320x212, but we can use --thumb-size 480x320 for sharper images. Here, we are using 480x320 as we are aiming for sharper images.

& "C:\Users\Pavilion\.platformio\penv\Scripts\python.exe" preprocessor\build_wiki_db.py --all-jpeg --thumb-size 480x320 "C:\Users\Pavilion\Downloads\wikipedia_en-simple_all_maxi_2026-09.zim" preprocessor\output_en_wiki

Once the above command is run, we can see the following logs in the terminal. It takes a decent amount of time to complete the task, as it depends on the zim model that we are preprocessing. Once this task is successful, it creates a folder named “output_en_wiki” under the preprocessor folder. We need the contents of this folder. You can edit the final directory name (here, it is output_en_wiki) to the correct name that matches the file that you have downloaded. In this case, it's an English version of the Wiki. That's why it's named “output_en_wiki”. It can be like “output_af_wiki” if we are using the Afrikaans version. After the process, open the same folder in your file explorer and copy all the contents of that folder. Make sure the microSD card has enough space to store all the data.

Preprocessor log while converting the Simple English ZIM file into ESP32-ready chunks.

Step 4⇒ Copy the Data to the microSD Card
Plug the microSD card (16 GB or 32 GB recommended) into your PC using a card reader. Format it as FAT32. Formatting a microSD card as FAT32 is easy. In Windows, right-click the drive > Format > FAT32.
Create a folder named wiki on the root of the SD card.
Copy all files from preprocessor\output_en_wiki\ into /wiki/, the folder that we need to create inside the microSD card’s root directory. The following are the files that are to be transferred to the folder named wiki on the SD Card.

The preprocessed index and chunk files that go into the /wiki folder on the microSD card.

After copying the files, the SD card folder structure should look something like the following. All the copied files are under a root folder named wiki.

SD Card Root/
└── wiki/
   ├── index.bin
   ├── sparse_index.bin
   ├── id_index.bin
   ├── index_meta.txt
   ├── articles_0000.dat
   ├── articles_0001.dat
   ...
   ├── img_index.bin
   ├── img_meta.txt
   ├── img_0000.dat
   ├── img_0001.dat

Insert the card into the built-in Micro SD Card slot on the ESP32-S3 CAM board and proceed to the next steps.
Step 5⇒ Flash the ESP32-S3 Web Server Firmware
Connect the ESP32-S3 CAM Module via USB and run the following command. Nothing needs to be changed; it will flash the firmware directly to the ESP32 Board.

cd firmware
$env:PYTHONIOENCODING="utf-8"
& "C:\Users\Pavilion\.platformio\penv\Scripts\pio.exe" run -e s3-cam-web -t upload

Flashing the firmware doesn’t take too much time, unlike when re-compressing the data. It shows the upload progress in the terminal window, just like how we can see the upload progress in Arduino IDE when we upload code to an Arduino / ESP32 Board.

PlatformIO confirms a successful upload of the offline Wikipedia web-server firmware.

Above is the success message that will come up once the firmware is flashed successfully. If we didn’t get a success message here, the most possible reason is the board being used shouldn’t be an ESP32-S3 board. This repo highly expects an ESP32-S3 board for successful operation.

Step 6⇒ Verify the SD Card Mount in the Serial Monitor
You can watch the boot output in the Serial Monitor to verify the mount:

& "C:\Users\Pavilion\.platformio\penv\Scripts\pio.exe" device monitor

You will see the following, which indicates everything is fine and a total of 129909 articles are available. These numbers vary depending on the file that is being used.

[wiki_db] Initialising storage...
[wiki_db] Setting custom SD_MMC pins: CLK=39, CMD=38, D0=40
[wiki_db] SD_MMC mounted successfully (1-bit mode)!
[wiki_db] Database ready: 129909 articles
[wiki_db] Images: 129909 available

Step 7⇒ Connect to the Hotspot and Browse Wikipedia Offline
Power on the ESP32-S3 CAM. On your phone, tablet, or laptop, open Wi-Fi settings and connect to the Wi-Fi network named Wikipedia-Offline that's hosted by the ESP32. It will be an open network, so there's no need to enter a password.

Joining the open Wikipedia-Offline Wi-Fi network created by the ESP32-S3.

Open any web browser and go to:
http://192.168.4.1/, which is the default fallback IP for AP mode in the ESP32 network stack.
You will be greeted with the full offline Wikipedia search engine, rich articles, and high-resolution images served directly from your ESP32-S3. (The Serial Monitor also remains active simultaneously for diagnostics!) As we start to search for something in the search bar, it will bring up related contents/articles. Clicking any item from the list will display the whole content of that article.

Result: Searching and Reading Wikipedia Offline on the ESP32

The following are some snapshots from the final results. As we can see, the initial window is one that says “Welcome to Offline Wikipedia”. It shows information like the total number of articles in the memory and all. As we start to type in a keyword, the system will show available articles related to the keyword. Once we click a keyword, the screen populates with the article's content. The content includes text and images. If we opt to use or download a “nopic” version, it will show text only and not any images.

Offline Wikipedia Running in the ESP32-S3 Board

Running Wikipedia Offline on an ESP32: Video Demo

Watch the complete ESP32 offline Wikipedia in action, running on an ESP32-S3 board with nothing more than a microSD card and its own Wi-Fi hotspot. The demo shows a phone joining the Wikipedia-Offline network, searching for a topic and opening a full article with images, all without an internet connection. It is a quick way to see the final result before you start building, and to check that your own setup behaves the same way.

Practical Uses of an Offline Wikipedia Server

∗ Emergency and survival kits: reference information when mobile networks are unavailable.
∗ Countryside classrooms and libraries: an encyclopedia accessible where internet is scarce or mobile data is too expensive.
∗ Field research: carrying out research in places without coverage.
∗ Learning about embedded systems: a project for hands-on learning about SD card technology and web servers using ESP32.

Conclusion

By bringing Kiwix’s open-source ZIM archives onto a low-cost microcontroller, this project proves that you don't need expensive server gear or active Wi-Fi to access global knowledge. Whether you're building an emergency survival kit, an off-grid educational tool, or just exploring the limits of microcontrollers, pairing the ESP32 with compressed offline data puts an entire library right in the palm of your hand. You can easily modify the repo and pair it with a TFT Display so that, instead of viewing the result on a web server, you can see it on the TFT Display. As an extension of this project, you can also integrate it with an offline speech system so the Wikipedia content is read aloud. We already have an article on building an ESP32 Text-to-Speech Offline System. Another option is to use some cloud-based services to run a text-to-speech system like our Text-to-Speech System using ESP32 and Wit.ai. For more builds like this, explore our ESP32 projects.

GitHub Repository

The complete source code for this ESP32 offline Wikipedia project is available in our GitHub repository, including the Python preprocessor that converts Kiwix ZIM files and the PlatformIO firmware for the ESP32-S3 web server. Clone or download the repository, follow the steps above, and you can have your own offline Wikipedia running in a single session.

ESP32 Offline Wikipedia GitHub RepisitoryESP32 Offline Wikipedia Downloadable Zip

Frequently Asked Questions About ESP32 Offline Wikipedia

⇥ What are Kiwix ZIM files?
ZIM files are file formats that have been compressed to enable offline web content to be read. Kiwix, which is charitable in nature, provides pre-prepared ZIM files in numerous languages on various topics. These files contain all the articles, images, and a search index, meaning that the internet is not required for reading.

⇥ Which ZIM file to use with the ESP32?
Select "simple", as it is smaller than the English Wikipedia. Select "maxi" when you want to view files with images, go for "nopic" whenever you wish to see only written articles or choose "mini" for articles that have summaries only. It is wise to check the size of your download and the capacity of your microSD card because its data must fit into the card.

⇥ What microSD card size is required?
This depends on the file you are going to convert. You can fit the Simple English version that has pictures on a 4GB card, while you will need higher capacity, such as 16 GB and 32 GB, for other compilations. A 2GB card may also work in some cases where only small articles are being converted.

⇥ Why do we have to format the microSD card as FAT32?
The microSD card will be used with the SD_MMC driver of the ESP32. This driver will only work with FAT32 format. Windows formatting utility only supports FAT32 formatting for cards up to 32GB of storage size. So, to format cards larger than that, special software needs to be downloaded and used. Also, thanks to the fact that the database is saved in small chunks, the limitation of 4GB for each file in FAT32 format won't be an issue.

Explore More AI-Powered Embedded Projects

Explore these DIY projects to build intelligent embedded systems using ESP32-CAM, AI-powered speech recognition, and Arduino-based text-to-speech. Learn to describe surroundings with an AI vision assistant, convert spoken words into text, and generate speech from text using simple hardware and practical step-by-step tutorials.

ESP32-CAM AI Vision Assistant Pendant for the Visually Impaired

ESP32-CAM AI Vision Assistant Pendant for the Visually Impaired

Here in this tutorial, we are going to explore an AI Vision Assistant for Visually Impaired People using an ESP32-CAM. A vision-based solution gives feedback based on the real surroundings and situations, which is more reliable than a proximity sensor-based solution.

ESP32 Speech-to-Text using Wit.ai

ESP32 Speech-to-Text using Wit.ai

In this project, we’ll be learning how to build an ESP32 Speech-to-Text system using an ESP32 development board. We’ll use an I2S MIC to record speech and an OLED display to display the converted text.

Arduino based Text to Speech (TTS) Converter

Arduino based Text to Speech (TTS) Converter

Today in this tutorial, we will learn how to make a text-to-speech converter using Arduino. Here in this Text-to-Speech converter, we’ve used an amplifier circuit to reduce the noise and get a clear sound. The amplifier circuit is made using the IC LM386.

Have any question related to this Article?

How Semtech Thinks About IoT Devices That Have to Last in the Field

Semtech’s Robert Comanescu, Vice President of R&D and Applications for Wireless ICs, and Tamir Reshef, Vice President of Products and Applications for the Analog and Mixed Signals business unit, discussed the challenges of building IoT devices that communicate reliably and withstand real-world conditions at electronica India and productronica India 2026 in Bengaluru.

Kikobot Robotics Went From a 3D-Printed Prototype to Industrial Robot Arms

Submitted by Staff on

Kikobot Robotics is an early-stage hardware startup in Pune. It began by building robotic arms for education and research and has since moved into industrial-grade arms and its own actuators. Raghvendra says the company builds robot arms for education, research and physical AI, and arms for industrial use cases as well. The lineup includes cobots, delta robots and gantry systems.

Getting Started with the PUI Audio AMP2X15 Audio Amplifier Board and Speakers

Adding a speaker to an embedded project is not always as simple as connecting it directly to a microcontroller or single-board computer. Most audio sources provide a relatively low-level signal, while a speaker needs more power to produce useful sound output. This is where an audio amplifier becomes important. The PUI Audio AMP2X15 is designed to bridge this gap, providing a compact two-channel amplification solution for driving speakers from a low-level audio source.

Peer Robotics Builds Collaborative Robots That Skip the Warehouse Retrofit

As part of our India Tech Tour series, we visited Peer Robotics' facility in Pune, where Ritwik walked us through the company's robots, manufacturing setup, and engineering processes. We also spoke with Ruchit, Vice President of Software Engineering at Peer Robotics, about the company's approach to deploying autonomous mobile robots in industrial environments.

Getting Started with ESP32-P4X-EYE: ESP32-P4 Vision Development Board

Vision development boards are useful when an embedded project needs to combine image capture, local display, audio input, storage and wireless connectivity in a single platform. The ESP32-P4X-EYE takes this approach around the ESP32-P4, adding an ESP32-C6 wireless module, a 2MP MIPI-CSI camera, a 1.54-inch SPI LCD, a digital microphone, MicroSD storage, USB interfaces and a 2 × 10-pin expansion header.

Qorvo's QPA9510, A Power Amplifier Supporting GSM, UHF, and FM Bands

Submitted by Staff on

Sub-1 GHz radio links sit at the core of a wide range of products, including public safety radios, smart utility meters, RFID readers, and vehicle telematics units. Many of these systems now need to work across several regional frequency allocations without a hardware redesign for each market, while still running efficiently on battery power in a compact footprint. Qorvo's QPA9510 power amplifier is built around that requirement.

The QPA9510 is a general-purpose amplifier covering 100 MHz to 1000 MHz, spanning GSM, E-GSM, UHF, and FM bands. Because the device supports this whole range while remaining tunable to specific sub-bands, a single design can be adapted for different regional frequency plans rather than starting from scratch for each one.

On performance, the QPA9510 delivers up to 34 dB of gain with more than 70 dB of analog gain control range, and it maintains linear operation up to a 1 dB compression point of roughly +35 dBm. It runs from a single supply (2.8 V to 4.8 V per its recommended operating conditions) and reaches efficiency of up to 55%. The part is fabricated on Qorvo's GaAs process and housed in a compact 3 mm x 3 mm QFN package, making it suitable for space-constrained designs.

To help designers evaluate the part, Qorvo also offers the QPA9510EVB evaluation board, tuned for the 865 MHz to 928 MHz range to assess GSM performance. The board includes the biasing and control circuitry needed to run the amplifier, along with RF input/output connectors, analog gain control, and single-supply operation, letting engineers test gain, output power, efficiency, and linearity without building supporting circuitry themselves.

Availability

Datasheets and ordering information for both the QPA9510 and the QPA9510EVB are available through Mouser.

Have any question related to this Article?

Running 260K Tiny-LLM on ESP32

Submitted by Anand D on

We are all facing a huge transition in tech due to AI. In projects, we can run AI models on PCs, SBCs or even Cloud servers. In this tutorial, we are trying to see if we can run an AI model on a microcontroller board like the ESP32. To test this out, we are going to deploy and benchmark the 260K-parameter Tiny-LLM by Andrej Karpathy using his open-source llama2.c Tiny-LLM framework across four distinct ESP32 boards. This model is designed to run in pure C. We will learn how to run the model on this memory-constrained microcontroller, examine the memory bandwidth limitations of SPI Flash streaming versus PSRAM, and evaluate real-world token generation speeds.

Hardware Test Bench

S noBoardFlashPSRAM
1ESP32 DevKit V14 MB0 MB
2ESP32-CAM4 MB4 MB
3Seeed XIAO ESP32-S38 MB8 MB
4ESP32-S3 DevKit (N16R8)16 MB8 MB

In the past, we have run .ino code on Arduino and ESP Boards, but in this case, it's slightly different. We need to flash this Tiny-LLM also onto the ESP32 Flash memory. Every ESP32 comes with Flash Memory, but it's optional that they have PSRAM. Depending on the project requirements. We need to buy the appropriate ESP32 Board. So, first of all. We are going to load the Tiny-LLM onto the Flash Memory of the ESP32. This has to be done no matter if the board has PSRAM or not. Let’s see where to go and download the model from.

Downloading the Tiny-LLM

First of all, we need to go to the Hugging Face page for Andrej Karpathy’s stories260L Tiny-LLM page and download the stories260K.bin and tok512.bin files which are by clicking the download option to their right side. Trained on the TinyStories dataset, this model outputs simple stories using the vocabulary of 3-to-4-year-olds. We cannot expect a high level of storytelling considering its size, which is around 1 MB.
 

Andrej Karpathy Hugging Face Page

Let's see what exactly these files do. The stories260K.bin is the raw binary file containing the actual learned weights of the 260K parameter model. stories260K.pt is the same stories260K.bin file, but in the PyTorch format. It contains the same mathematical weights as the .bin file. Its use is to load the model into Python code to fine-tune it or analyze it using PyTorch code. For our tutorial, we are not going to use it. The tok512.bin is a custom binary version of the tokeniser. It tells the C program (run.c), which we will run in Arduino IDE, how to map numeric IDs into English letters and words. tok512.model is used by some Python text-processing libraries to break down sentences into tokens before training. We are not going to use that as well. As of now, let's download the stories260K.bin and tok512.bin files for the tutorial.

Flashing Tiny-LLM to ESP32 Flash Memory

Before flashing the model, let's see what the folder structure should look like for this to work. We have the ino code and a run.c code. In the same directory where these code files reside, create a folder named data and save these models, which we downloaded just now, to the data folder. This is what my folder structure looks like. These files are also provided in the GitHub repo provided below.
 

Code Folder Structure

Once this is set up, let's go ahead and flash the models in the data folder to the ESP32 Flash. By default, Arduino IDE only allows us to upload code (.ino files). So, we need an additional tool to flash these files to the ESP32. The Arduino LittleFS Upload Plugin is a tool that allows US to store non-code files on our local PC (like web pages, images, or configuration files) directly onto the built-in flash memory of microcontrollers like  ESP8266, ESP32, or RP2040.

We need to go to the official Earle Philhower LittleFS Upload Releases page and download the file named arduino-littlefs-upload-1.6.3.vsix. This is the latest version of the compiled release file at the time of writing this tutorial. Copy the downloaded file and paste it into “C:\Users\User name\.arduinoIDE\plugins”. If it gives any error when executing, then keep the file directly within the “C:\Users\Pavilion\.arduinoIDE” directory.

Uploading Models To Flash

For flashing the Model weights to Flash Memory, use the shortcut Ctrl + Shift + P and click Upload LittleFS to Pico/ESP8266/ESP32. Wait for the LittleFS Upload terminal window to turn to “Completed Upload”. Once this is done, we have successfully loaded the model weights to the ESP32 Flash Memory. If any error comes up, the most likely reason is improper folder structure or folder naming. Just make sure that part is done properly as mentioned above. Once this is done, we are ready to upload the code to the ESP32.

Arduino IDE Configurations

Check the following configurations in the Tools menu in Arduino IDE as shown below. The options must be selected as per the RAM and Flash size of the specific ESP32 board you are using. If it does not have PSRAM, just keep the PSRAM option disabled. If it has PSRAM, select the PSRAM size. In the screenshot, it shows the settings for an ESP32 S3 Dev board with 16 MB Flash and 8 MB PSRAM on the right and a normal ESP32 Dev Board with the default 4 MB Flash and no PSRAM on the left. The right configuration for your board will make this work.

Arduino IDE Configurations

From the GitHub link below, all the code and the models are available for download. One thing to note is that the run.c code to be flashed on an ESP32 Dev board that does not have any PSRAM is kept in a separate folder named “run.c code for ESP32 without PSRAM” in the repo. Use that run.c if the ESP32 does not have any PSRAM; otherwise, use the one in the main directory. After setting the right configurations. Hit the upload button. The code will be successfully flashed onto your ESP32.

Code Explanation

Let's take a look at the coding part of this tutorial.

#include "FS.h"
#include "LittleFS.h"
// Expose C functions from run.c to C++ compiler
extern "C" {
 void run_llama(const char* model_path, const char* tokenizer_path, float temperature, float topp, int steps, const char* prompt);
}

Imports the flash file system libraries required to read files directly from the ESP32’s flash chip. extern "C" { ... } allows the C++ sketch to link directly to run_llama(), which is written in plain C inside the companion run.c file.

void llamaTask(void *pvParameters) {
 Serial.println("\n--- Starting LittleFS Initialization ---");
 if (!LittleFS.begin(false)) {
   Serial.println("LittleFS Mount Failed! Check if filesystem was uploaded.");
   vTaskDelete(NULL);
   return;
 }
 Serial.println("LittleFS Mounted Successfully.");
 
 if (!LittleFS.exists("/stories260K.bin")) {
   Serial.println("Error: /stories260K.bin not found on LittleFS!");
   vTaskDelete(NULL);
   return;
 }
 
 if (!LittleFS.exists("/tok512.bin")) {
   Serial.println("Error: /tok512.bin not found on LittleFS!");
   vTaskDelete(NULL);
   return;
 }

void llamaTask() defines the body of the FreeRTOS background task. LittleFS.begin(false) mounts the SPI Flash file system. Passing false ensures it won't auto-format the storage if mounting fails, protecting existing files. LittleFS.exists() checks whether the two essential binary files that we downloaded into the folder named ‘data’ exist on flash. So make sure they exist, or else it gives an error. vTaskDelete(NULL) safely terminates and cleans up this task if storage mounting or file verification fails.

void setup() {
 Serial.begin(115200);
 delay(2000); // Allow hardware serial connection to settle
 Serial.println("==========================================");
 Serial.println("   ESP32-S3 Tiny Llama Inference Engine   ");
 Serial.println("==========================================");
 // Non-blocking memory mode log
 if (ESP.getPsramSize() == 0) {
   Serial.println("ℹ️ PSRAM not detected. Running in Flash-Streaming mode.");
 } else {
   Serial.printf("PSRAM Available: %d Bytes\n", ESP.getFreePsram());
 }

Serial.begin(115200) initializes the primary hardware serial line at 115,200 baud. ESP.getPsramSize() checks if external PSRAM is attached to the ESP32-S3. If none is found, it notes that the system will stream weights directly out of Flash memory.

void loop() {
 // Read typed text from the Serial Monitor
 if (Serial.available() > 0) {
   String inputString = Serial.readStringUntil('\n');
   inputString.trim(); // Strip carriage returns and spaces
   if (inputString.length() > 0) {
     char promptBuffer[256];
     inputString.toCharArray(promptBuffer, sizeof(promptBuffer));
     // Push user text into queue for llamaTask to consume
     xQueueSend(promptQueue, &promptBuffer, pdMS_TO_TICKS(100));
   }
 }
 vTaskDelay(pdMS_TO_TICKS(50)); // Poll serial smoothly
}

Serial.available() > 0 checks if the user has typed text into the Arduino Serial Monitor. Serial.readStringUntil('\n') & .trim() reads characters until a newline is reached and strips out extra whitespace or carriage returns (\r). inputString.toCharArray() converts the Arduino C++ String object into a plain C-style char array buffer. xQueueSend() pushes the new prompt into promptQueue with a 100 ms timeout. Core 1 instantly picks up this message to run the next inference. vTaskDelay(pdMS_TO_TICKS(50)) pauses Core 0 execution briefly to keep the processor watchdog timer happy and yield control back to background OS routines.

Results

After the code is successfully uploaded, open the serial monitor. It will start generating its default story the moment Arduino IDE detects the board. The code is made in such a way that we can give a starting prompt for the story in the serial monitor, and the model generates a continuing story. But, practically, it doesn't work like that. It’s just a 1 MB tiny LLM that doesn't perform like a real LLM. But at least it generates words and outputs the tokens/second in the serial monitor. We can keep this as a starting point for running tiny LLMs on microcontroller boards.

Speed Results Of The Model In ESP32 Boards

Above are the results of the model running on 4 different ESP32-based boards. The Board names and respective speeds are highlighted in the image above and listed in the table below.

S no           BoardFlashPSRAMSpeed (tok/sec)
1ESP32 DevKit V14 MB0 MB2.33 tok/sec
2ESP32-CAM4 MB4 MB11.12 tok/sec
3Seeed XIAO ESP32-S38 MB8 MB21.41 tok/sec
4ESP32-S3 DevKit (N16R8)16 MB8 MB22.02 tok/sec

Conclusion

This benchmark confirms that PSRAM bandwidth is the single most critical factor for running local Generative AI models at the edge. While running on Flash enables low-cost non-PSRAM hardware to execute basic inference tasks (2.33 tok/sec), enabling Octal-SPI PSRAM on the ESP32-S3 unlocks nearly a 10x performance gain (22.02 tok/sec). This opens up practical possibilities for TinyML engineers to deploy lightweight, fully offline Micro-LLMs for local intent classification, offline smart home interfaces, and real-time edge processing without sending data to external cloud APIs.

GitHub Repository 
Running 260K Tiny-LLM on ESP32 GitHub Running 260K Tiny-LLM on ESP32 Downloadable Zip

Have any question related to this Article?

JioFind 4G GPS Tracker Teardown: A Complete Inside-Out Hardware Breakdown

Bluetooth trackers like Apple AirTags and Samsung SmartTag are a convenient way to locate misplaced items nearby, but their dependence on nearby smartphones limits their usefulness over long distances. Cellular GPS trackers overcome this limitation by communicating directly over the mobile network, allowing them to report their location from anywhere with network coverage. This makes them useful for tracking vehicles, luggage, school bags, pets, and other valuable belongings.