Human Following Robot Using Arduino and Ultrasonic Sensor

Submitted by Gourav Tak on

Working of Human Following Robot Using Arduino

In recent years, robotics has witnessed significant advancements, enabling the creation of intelligent machines that can interact with the environment. One exciting application of robotics is the development of human-following robots. These robots can track and follow a person autonomously, making them useful in various scenarios like assistance in crowded areas, navigation support, or even as companions. In this article, we will explore in detail how to build a human following robot using Arduino and three ultrasonic sensors, complete with circuit diagrams and working code. Also, check all the Arduino-based Robotics projects by following the link.

The working of a human following robot using Arduino code and three ultrasonic sensors is an interesting project. What makes this project particularly interesting is the use of not just one, but three ultrasonic sensors. This adds a new dimension to the experience, as we typically see humans following a robot built with one ultrasonic, two IR, and one servo motor.  This servo motor has no role in the operation and also adds unnecessary complications. So I removed this servo and the IR sensors and used 3 ultrasonic sensors. With ultrasonic sensors, you can measure distance and use that information to navigate and follow a human target. Here’s a general outline of the steps involved in creating such a robot.

 

 

Components Needed for Human Following Robot Using Arduino

  • Arduino UNO board ×1

  • Ultrasonic sensor ×3

  • L298N motor driver ×1

  • Robot chassis

  • BO motors ×2

  • Wheels ×2

  • Li-ion battery 3.7V ×2

  • Battery holder ×1

  • Breadboard

  • Ultrasonic sensor holder ×3

  • Switch and jumper wires

Human Following Robot Using Arduino Circuit Diagram

Here is the schematic diagram of a Human-following robot circuit.

Arduino Human Following Robot Circuit Diagram

This design incorporates three ultrasonic sensors, allowing distance measurements in three directions front, right, and left. These sensors are connected to the Arduino board through their respective digital pins. Additionally, the circuit includes two DC motors for movement, which are connected to an L298N motor driver module. The motor driver module is, in turn, connected to the Arduino board using its corresponding digital pins. To power the entire setup, two 3.7V li-ion cells are employed, which are connected to the motor driver module via a switch.

Overall, this circuit diagram showcases the essential components and connections necessary for the Human-following robot to operate effectively.

arduino human following robot circuit

Circuit Connection:

Arduino and HC-SR04 Ultrasonic Sensor Module:

HC-SR04 Ultrasonic sensor Module

  • Connect the VCC pin of each ultrasonic sensor to the 5V pin on the Arduino board.

  • Connect the GND pin of each ultrasonic sensor to the GND pin on the Arduino board.

  • Connect the trigger pin (TRIG) of each ultrasonic sensor to separate digital pins (2,4, and 6) on the Arduino board.

  • Connect the echo pin (ECHO) of each ultrasonic to separate digital pins (3,5, and 7) on the Arduino board.

Arduino and Motor Driver Module:

  • Connect the digital output pins of the Arduino (digital pins 8, 9, 10, and 11) to the appropriate input pins (IN1, IN2, IN3, and IN4) on the motor driver module.

  • Connect the ENA and ENB pins of the motor driver module to the onboard High state pin with the help of a female header.

  • Connect the OUT1, OUT2, OUT3, and OUT4 pins of the motor driver module to the appropriate terminals of the motors.

  • Connect the VCC (+5V) and GND pins of the motor driver module to the appropriate power (Vin) and ground (GND) connections on the Arduino.

Power Supply:

  • Connect the positive terminal of the power supply to the +12V input of the motor driver module.

  • Connect the negative terminal of the power supply to the GND pin of the motor driver module.

  • Connect the GND pin of the Arduino to the GND pin of the motor driver module.

Human Following Robot Using Arduino Code

Here is a simple 3 Ultrasonic sensor-based Human following robot using Arduino Uno code that you can use for your project.

Ultrsonic Sensors on Robot

This code reads the distances from three ultrasonic sensors (‘frontDistance’, ‘leftDistance’, and ‘rightDistance’). It then compares these distances to determine the sensor with the smallest distance. If the smallest distance is below the threshold, it moves the car accordingly using the appropriate motor control function (‘moveForward()’, ‘turnLeft()’, ‘turnRight()’). If none of the distances are below the threshold, it stops the motor using ‘stop()’.

In this section, we define the pin connections for the ultrasonic sensors and motor control. The S1Trig, S2Trig, and S3Trig, variables represent the trigger pins of the three ultrasonic sensors, while S1Echo, S2Echo, and S3Echo, represent their respective echo pins.

The LEFT_MOTOR_PIN1, LEFT_MOTOR_PIN2, RIGHT_MOTOR_PIN1, and RIGHT_MOTOR_PIN2 variables define the pins for controlling the motors.

The MAX_DISTANCE and MIN_DISTANCE_BACK variables set the thresholds for obstacle detection.

// Ultrasonic sensor pins
#define S1Trig 2
#define S2Trig 4
#define S3Trig 6
#define S1Echo 3
#define S2Echo 5
#define S3Echo 7
// Motor control pins
#define LEFT_MOTOR_PIN1 8
#define LEFT_MOTOR_PIN2 9
#define RIGHT_MOTOR_PIN1 10
#define RIGHT_MOTOR_PIN2 11
// Distance thresholds for obstacle detection
#define MAX_DISTANCE 40
#define MIN_DISTANCE_BACK 5

Make sure to adjust the values of ‘MIN_DISTANCE_BACK’ and ‘MAX_DISTANCE’ according to your specific requirements and the characteristics of your robot.

The suitable values for ‘MIN_DISTANCE_BACK’ and ‘MAX_DISTANCE’ depend on the specific requirements and characteristics of your human-following robot. You will need to consider factors such as the speed of your robot, the response time of the sensors, and the desired safety margin

Here are some general guidelines to help you choose suitable values.

MIN_DISTANCE_BACK’ This value represents the distance at which the car should come to a stop when an obstacle or hand is detected directly in front. It should be set to a distance that allows the car to back safely without colliding with the obstacle or hand. A typical value could be around 5-10 cm.

MAX_DISTANCE’ This value represents the maximum distance at which the car considers the path ahead to be clear and can continue moving forward. It should be set to a distance that provides enough room for the car to move without colliding with any obstacles or hands. If your hand and obstacles are going out of this range, the robot should be stop. A typical value could be around 30-50 cm.

These values are just suggestions, and you may need to adjust them based on the specific characteristics of your robot and the environment in which it operates.

These lines set the motor speed limits. ‘MAX_SPEED’ denotes the upper limit for motor speed, while ‘MIN_SPEED’ is a lower value used for a slight left bias. The speed values are typically within the range of 0 to 255, and can be adjusted to suit our specific requirements.

// Maximum and minimum motor speeds
#define MAX_SPEED 150
#define MIN_SPEED 75

The ‘setup()’ function is called once at the start of the program. In the setup() function, we set the motor control pins (LEFT_MOTOR_PIN1, LEFT_MOTOR_PIN2, RIGHT_MOTOR_PIN1, RIGHT_MOTOR_PIN2) as output pins using ‘pinMode()’ . We also set the trigger pins (S1Trig, S2Trig, S3Trig) of the ultrasonic sensors as output pins and the echo pins (S1Echo, S2Echo, S3Echo) as input pins. Lastly, we initialize the serial communication at a baud rate of 9600 for debugging purposes.

void setup() {
  // Set motor control pins as outputs
  pinMode(LEFT_MOTOR_PIN1, OUTPUT);
  pinMode(LEFT_MOTOR_PIN2, OUTPUT);
  pinMode(RIGHT_MOTOR_PIN1, OUTPUT);
  pinMode(RIGHT_MOTOR_PIN2, OUTPUT);
  //Set the Trig pins as output pins
  pinMode(S1Trig, OUTPUT);
  pinMode(S2Trig, OUTPUT);
  pinMode(S3Trig, OUTPUT);
  //Set the Echo pins as input pins
  pinMode(S1Echo, INPUT);
  pinMode(S2Echo, INPUT);
  pinMode(S3Echo, INPUT);
  // Initialize the serial communication for debugging
  Serial.begin(9600);
}

This block of code consists of three functions (‘sensorOne()’, ‘sensorTwo()’, ‘sensorThree()’) responsible for measuring the distance using ultrasonic sensors.

The ‘sensorOne()’ function measures the distance using the first ultrasonic sensor. It's important to note that the conversion of the pulse duration to distance is based on the assumption that the speed of sound is approximately 343 meters per second. Dividing by 29 and halving the result provides an approximate conversion from microseconds to centimeters.

The ‘sensorTwo()’ and ‘sensorThree()’ functions work similarly, but for the second and third ultrasonic sensors, respectively.

// Function to measure the distance using an ultrasonic sensor
int sensorOne() {
  //pulse output
  digitalWrite(S1Trig, LOW);
  delayMicroseconds(2);
  digitalWrite(S1Trig, HIGH);
  delayMicroseconds(10);
  digitalWrite(S1Trig, LOW);
  long t = pulseIn(S1Echo, HIGH);//Get the pulse
  int cm = t / 29 / 2; //Convert time to the distance
  return cm; // Return the values from the sensor
}
//Get the sensor values
int sensorTwo() {
  //pulse output
  digitalWrite(S2Trig, LOW);
  delayMicroseconds(2);
  digitalWrite(S2Trig, HIGH);
  delayMicroseconds(10);
  digitalWrite(S2Trig, LOW);
  long t = pulseIn(S2Echo, HIGH);//Get the pulse
  int cm = t / 29 / 2; //Convert time to the distance
  return cm; // Return the values from the sensor
}
//Get the sensor values
int sensorThree() {
  //pulse output
  digitalWrite(S3Trig, LOW);
  delayMicroseconds(2);
  digitalWrite(S3Trig, HIGH);
  delayMicroseconds(10);
  digitalWrite(S3Trig, LOW);
  long t = pulseIn(S3Echo, HIGH);//Get the pulse
  int cm = t / 29 / 2; //Convert time to the distance
  return cm; // Return the values from the sensor
}

In this section, the ‘loop()’ function begins by calling the ‘sensorOne()’, ‘sensorTwo()’, and ‘sensorThree()’ functions to measure the distances from the ultrasonic sensors. The distances are then stored in the variables ‘frontDistance’, ‘leftDistance’, and ‘rightDistance’.

Next, the code utilizes the ‘Serial’ object to print the distance values to the serial monitor for debugging and monitoring purposes.

void loop() {
  int frontDistance = sensorOne();
  int leftDistance = sensorTwo();
  int rightDistance = sensorThree();
  Serial.print("Front: ");
  Serial.print(frontDistance);
  Serial.print(" cm, Left: ");
  Serial.print(leftDistance);
  Serial.print(" cm, Right: ");
  Serial.print(rightDistance);
  Serial.println(" cm");

In this section of code condition checks if the front distance is less than a threshold value ‘MIN_DISTANCE_BACK’ that indicates a very low distance. If this condition is true, it means that the front distance is very low, and the robot should move backward to avoid a collision. In this case, the ‘moveBackward()’ function is called.

if (frontDistance < MIN_DISTANCE_BACK) {
    moveBackward();
    Serial.println("backward");

If the previous condition is false, this condition is checked. if the front distance is less than the left distance, less than the right distance, and less than the ‘MAX_DISTANCE’ threshold. If this condition is true, it means that the front distance is the smallest among the three distances, and it is also below the maximum distance threshold. In this case, the ‘moveForward()’ function is called to make the car move forward.

else if (frontDistance < leftDistance && frontDistance < rightDistance && frontDistance < MAX_DISTANCE) {
    moveForward();
    Serial.println("forward");

If the previous condition is false, this condition is checked. It verifies if the left distance is less than the right distance and less than the ‘MAX_DISTANCE’ threshold. This condition indicates that the left distance is the smallest among the three distances, and it is also below the minimum distance threshold. Therefore, the ‘turnLeft()’ function is called to make the car turn left.

else if (leftDistance < rightDistance && leftDistance < MAX_DISTANCE) {
    turnLeft();
    Serial.println("left");

If neither of the previous conditions is met, this condition is checked. It ensures that the right distance is less than the ‘MAX_DISTANCE’ threshold. This condition suggests that the right distance is the smallest among the three distances, and it is below the minimum distance threshold. The ‘turnRight()’ function is called to make the car turn right.

else if (rightDistance < MAX_DISTANCE) {
    turnRight();
    Serial.println("right");

If none of the previous conditions are true, it means that none of the distances satisfy the conditions for movement. Therefore, the ‘stop()’ function is called to stop the car.

 else {
    stop();
    Serial.println("stop");

In summary, the code checks the distances from the three ultrasonic sensors and determines the direction in which the car should move based on the 3 ultrasonic sensors with the smallest distance.

 

Important aspects of this Arduino-powered human-following robot project include:

  • Three-sensor setup for 360-degree human identification
  • Distance measurement and decision-making in real-time
  • Navigation that operates automatically without human assistance
  • Avoiding collisions and maintaining a safe following distance

 

 

Technical Summary and GitHub Repository 

Using three HC-SR04 ultrasonic sensors and an L298N motor driver for precise directional control, this Arduino project shows off the robot's ability to track itself. For simple replication and modification, the full source code, circuit schematics, and assembly guidelines are accessible in our GitHub repository. To download the Arduino code, view comprehensive wiring schematics, and participate in the open-source robotics community, visit our GitHub page.

Code Schematics Download Icon

 

Frequently Asked Questions

⇥ How does an Arduino-powered human-following robot operate?
Three ultrasonic sensors are used by the Arduino-powered human following robot to determine a person's distance and presence. After processing this data, the Arduino manages motors to follow the identified individual while keeping a safe distance.

⇥ Which motor driver is ideal for an Arduino human-following robot?
The most widely used motor driver for Arduino human-following robots is the L298N. Additionally, some builders use the L293D motor driver shield, which connects to the Arduino Uno directly. Both can supply enough current for small robot applications and manage 2-4 DC motors.

⇥ Is it possible to create a human-following robot without soldering?
Yes, you can use motor driver shields that connect straight to an Arduino, breadboards, and jumper wires to construct a human-following robot. For novices and prototyping, this method is ideal.

⇥ What uses do human-following robots have in the real world?
Shopping cart robots in malls, luggage-carrying robots in airports, security patrol robots, elderly care assistance robots, educational demonstration robots, and companion robots that behave like pets are a few examples of applications.

 

Conclusion

This human following robot using Arduino project and three ultrasonic sensors is an exciting and rewarding project that combines programming, electronics, and mechanics. With Arduino’s versatility and the availability of affordable components, creating your own human-following robot is within reach.

Human-following robots have a wide range of applications in various fields, such as retail stores, malls, and hotels, to provide personalized assistance to customers. Human-following robots can be employed in security and surveillance systems to track and monitor individuals in public spaces. They can be used in Entertainment and events, elderly care, guided tours, research and development, education and research, and personal robotics.

They are just a few examples of the applications of human-following robots. As technology advances and robotics continues to evolve, we can expect even more diverse and innovative applications in the future.

Explore Practical Projects Similar To Robots Using Arduino

Explore a range of hands-on robotics projects powered by Arduino, from line-following bots to obstacle-avoiding vehicles. These practical builds help you understand sensor integration, motor control, and real-world automation techniques. Ideal for beginners and hobbyists, these projects bring theory to life through interactive learning.

 Simple Light Following Robot using Arduino UNO

Simple Light Following Robot using Arduino UNO

Today, we are building a simple Arduino-based project: a light-following robot. This project is perfect for beginners, and we'll use LDR sensor modules to detect light and an MX1508 motor driver module for control. By building this simple light following robot you will learn the basics of robotics and how to use a microcontroller like Arduino to read sensor data and control motors.

Line Follower Robot using Arduino UNO: How to Build (Step-by-Step Guide)

Line Follower Robot using Arduino UNO: How to Build (Step-by-Step Guide)

This step-by-step guide will show you how to build a professional-grade line follower robot using Arduino UNO, with complete code explanations and troubleshooting tips. Perfect for beginners and intermediate makers alike, this project combines hardware interfacing, sensor calibration, and motor control fundamentals.

Have any question related to this Article?

ESP32 Camera Dev Board Comparison: ESP32-CAM vs ESP32-S3-CAM vs XIAO Sense-Which Should You Choose?

Submitted by Anand D on

Introduction

ESP32 Cameras have become a popular choice for building low-cost wireless cameras, smart surveillance systems, and even small AI projects. But today, there are several options available, from the original ESP32-CAM to newer ESP32-S3-based camera boards and compact boards like the XIAO ESP32-S3 Sense. At first glance, they may look similar, but there are some major differences in their processing power, memory, camera sensors, size, and AI capabilities. So, which one should you pick for your next project? In this article, we will compare the ESP32-CAM, ESP32-S3-CAM, and XIAO ESP32-S3 Sense and see which camera board fits in for different types of projects.

ESP32 CAM

ESP32 CAM Development Module

The ESP32-CAM should be the most familiar ESP32 camera board. It is based on the original ESP32 and usually comes with an OV2640, the 2 MP camera from OmniVision, 4 MB Flash, 4MB PSRAM, a microSD card slot, Wi-Fi, and Bluetooth. Its biggest advantage is its simplicity and low cost. The ESP32-CAM can capture images, stream video over Wi-Fi, store photographs on a microSD card, and communicate with a server or cloud service. This makes it a good choice for projects such as Wi-Fi security cameras, Motion detection cameras, Remote monitoring, Simple object detection, etc.

Now, let's see some of the major limitations that come with this board. It just has 4 MB of PSRAM, which is useful for camera frame buffers, but newer ESP32-S3 boards generally provide more memory and are better suited for running image-processing or AI workloads locally. Another inconvenience is programming. Most ESP32-CAM boards do not have a USB interface built into the board, so we will have to use an ESP32 CAM programming module or any of the widely available USB-TTL Converter Modules. Choose ESP32-CAM if the requirement is a cheap and simple Wi-Fi camera and you don't need heavy local AI processing.

ESP32-S3-CAM

ESP32-S3 Development Module

The ESP32-S3-CAM takes advantage of the newer ESP32-S3 chip. The ESP32-S3 uses a dual-core Xtensa LX7 processor running at up to 240 MHz. While the clock speed is similar to the original ESP32, the S3 has several architectural improvements that make it more suitable for applications involving signal processing, machine learning, and computer vision. Many ESP32-S3 camera boards also come with 8 MB of PSRAM, giving the camera application more room for frame buffers and larger programs. This becomes important when you move from a simple camera project to something that involves running an AI model with some image processing and all.

For example, an ESP32-S3 camera can be used for Face detection, Object detection, Image classification, Person detection, TinyML projects, AI-enabled IoT devices, etc.  The ESP32-S3 also comes with a USB port, which makes programming and USB-based applications easier on boards that expose the interface. Typically, an ESP32-S3 CAM comes with either 16 MB Flash and 8MB PSRAM or 8 MB Flash and 8MB PSRAM. Also, they come from various 3rd party manufacturers as well. These boards can even support the latest OmniVision 5MP OV5640 sensor as well. Normally, they come with the OV3660 Sensor, the 3 MP sensor from the manufacturer. So, always check the exact specifications before buying one.
We have a detailed article that teaches how to identify the PSRAM and Flash size of your official ESP32 Modules by checking the Specification Identifier Code on them. It also talks about the memory structure and different types of memory that we can see in an ESP32 Module.

XIAO ESP32-S3 Sense

XIAO ESP32-S3 Board With Expansion Board and Camera Sensor

If the ESP32-S3-CAM is designed for more advanced camera applications, the SeedStudio XIAO ESP32-S3 Sense takes a different approach. It focuses on size and integration.
The XIAO ESP32-S3 Sense combines the ESP32-S3 with a camera, microphone, microSD card support, USB connectivity and even a PMIC in a very small package.

The current version uses an OV3660 camera sensor, which offers up to 3 MP resolution. It also comes with 8 MB PSRAM and 8 MB Flash, which is double the capacity of the normal ESP32-CAM module. The Sense also includes a digital microphone, which means you can build a device that can both see and hear without needing a separate audio board. Some example projects that can be done with the XIAO ESP32-S3 Sense are Voice-controlled cameras, AI vision assistants, Smart home sensors, Wearable devices, Voice + vision AI projects, etc. The possibilities are endless due to their processing power and compact size.

The XIAO ESP32-S3 board itself measures only around 21 × 17.8 mm, making it much easier to fit into compact projects. The trade-off is GPIO availability. Because the camera and other onboard hardware use several pins, you don't get the same freedom as you might with a larger ESP32-S3 development board. Choose XIAO ESP32-S3 Sense if the requirement is the smallest package possible and you need camera, audio, storage, and AI capabilities in one board.

Which One Has the Best Image Quality?

 ESP32 Vs ESP32 CAM Vs XIAO ESP32 S3 ALL Together

This is where things get a little more complicated. The ESP32 itself doesn't determine the camera's image quality. The camera sensor and lens also play a major role. The classic ESP32-CAM normally uses the OV2640, which has a resolution of up to 2 MP.
The XIAO ESP32-S3 Sense currently uses the OV3660, which can capture up to 3 MP.
Some ESP32-S3-CAM boards use the OV3660, while others may use higher-resolution sensors such as the OV5640. So you shouldn't assume that every ESP32-S3-CAM will have a better camera simply because it uses an ESP32-S3.

The underlying fact is that an ESP32 S3 board, in any way, is not able to do a lot of computer vision applications and run heavy models on it. So, the question is there a need to use a high-quality 5MP sensor on it? An ESP32 S3 with 16 MB Flash and 8 MB PSRAM can do a lot of great jobs with a 3 MP Sensor.

Which One Has the Best Processing Power?

PSRAM and Flash are the ones that make all the difference. When working with camera projects, PSRAM is extremely useful. A camera frame can require a significant amount of memory, especially when using higher resolutions or multiple frame buffers. If it doesn't have any PSRAM, the internal SRAM, which is typically around 520 KB, will not be sufficient to run inferences or do some complex matrix multiplications. So, PSRAM plays a major role if we are leveraging the ESP32 capabilities for AI-based applications.

Another option that is possible but not an ideal practice is that you can run inferences or do the AI workloads on the Flash memory itself, but that is not very efficient, as Flash is not meant for computations and all, but it can store the AI Models, the firmware, code, and stuff like that. But for AI processing or creating buffers for processing, PSRAM is a must. We have tried running a tiny LLM from PSRAM as well as Flash. The results were astonishing. When it was run from PSRAM, we got 22 tokens per second, while when we ran it from Flash Memory, the speed was merely 2 tokens per second. More PSRAM gives your application more room for camera frame buffers, audio buffers, AI models, image processing, etc.

The typical configurations are:

  • ESP32-CAM - 4MB Flash, 4 MB PSRAM

  • ESP32-S3-CAM - 16 MB Flash, 8 MB PSRAM

  • XIAO ESP32-S3 Sense - 8 MB Flash, 8 MB PSRAM

What About AI?

This is probably the biggest reason to consider an ESP32-S3. The original ESP32-CAM is perfectly capable of capturing an image and sending it somewhere else, like a cloud server for processing. But if you want the ESP32 itself to perform more of the processing, the ESP32-S3 is the better platform.

Cloud AI Vs Local AI In ESP32

 

The S3 is designed to accelerate certain vector and signal-processing operations, which are useful for machine-learning workloads. However, it is important to set expectations. An ESP32-S3 is not a replacement for a Raspberry Pi or a Jetson for large computer-vision models. It is best suited to small, optimised edge-AI models.

Conclusion

Below is the overall summary in table format.

ESP32 Model                   Application
ESP32-CAMBest for simple and affordable camera projects
ESP32-S3-CAMBest for AI and advanced vision
XIAO ESP32-S3 SenseBest for compact AI, vision, and audio projects

If you only need a simple camera, the original ESP32-CAM is still a great choice. If you want to experiment with AI and computer vision, I would go with an ESP32-S3-CAM. But if you want to build a tiny AI device that can see, hear, store data, and connect wirelessly, the XIAO ESP32-S3 Sense is probably the most versatile option of the three.

Have any question related to this Article?

How to Identify ESP32 Flash & PSRAM Size

Submitted by Anand D on

ESP32 development boards have been around for years, and we all have been doing a lot of projects with them. They come in various models, with different peripherals, and they perform different levels of tasks. If we are to check into a simple example, we have the normal ESP32 WROOM development boards as well as the same ESP32 with a camera sensor mounted onboard, the ESP32-CAM. The same way, we have the board with an onboard MIC, much more processing power, different power consumption, and memory capacities.

We pick the ESP32 modules for various projects based on their memory capacities, like ESP32 flash memory size, PSRAM, cores, power consumption, etc... In this tutorial, we will see how the ESP32 memory is classified and see the most easy and straight forward method of identifying the Flash and PSRAM size of an ESP32 module. But before that, let's understand the ESP32 Memory Architecture with the block diagram below.

ESP32 Memory Architecture

ESP32 Memory Architecture Block Diagram

Let's explore the memory classification. All of the ESP32s fall under one combination as per this architecture. Basically, the ESP32’s memory is classified into Internal and External. In the Internal Memory, we have the ROM and SRAM(Static RAM). The ROM stores permanent code such as the bootloader/ROM routines. The SRAM is the main working memory. The ESP32 SRAM size matters most for real-time performance, since it is used by our applications, FreeRTOS, Wi-Fi/Bluetooth stacks, buffers, etc.

A standard ESP32 development module comes with 4MB of Flash. It stores our program/code/firmware, files, web pages, configuration data, etc. Unlike SRAM, it retains data when power is removed. Our ESP32 runs the code whenever we power it on, right? That is due to this Flash Memory. We can see ESP32 modules with 2MB, 4MB, an ESP32 PSRAM 8mb varian, 16MB and 32MB Flash variants. Then we have PSRAM (Pseudo Static RAM). It's an additional RAM connected externally to the ESP32. Common sizes include 2 MB, 4 MB and 8 MB, depending on the module. It is mainly used in applications involving video streaming, image processing, AI, LVGL and large buffers. Now. Let's try to learn how we can identify the Flash and PSRAM Size of an ESP32 Development Module.

RF Shield Labelling

This is how Espressif Systems, the official manufacturer of the ESP32 chips and modules, classifies their products. On the RF shielding of the ESP32 WROOM modules, we can see some details that describe what's inside. They are the Espressif logo, the Module Name, the Certification ID that indicates the certification this module has passed, the Company Name, usually Espressif Systems (Shanghai) Co., Ltd, mostly written in Chinese, and a Data Matrix scanning which returns an 18-character code that conveys the Production Date Code and the Module MAC ID. Then we have the Specification Identifier, which tells a lot of details about the module. Let's try to understand the Specification Identifier Code and learn to identify the Flash and PSRAM of an ESP32 Module.

Specification Identifier Code in ESP32

The Specification Identifier is defined by Espressif to indicate the product status, operating temperature, and the memory capacity of Espressif modules.

ESP32 Spec Identifier Code

The above details are clearly shown below in a table format so that we can easily understand them.

StatusTemperatureFlashPSRAMReserved
XXN: 85 °C/65 °C2: 2 MBR2: 2 MBXX
MNH: 105 °C4: 4 MBR8: 8 MB 
  8: 8 MB  
  16: 16 MB  
  32: 32 MB  

As seen above, in the XX/MN, the first two prefix characters identify the product status; N indicates the operating temperature is 85 °C/65 °C, and H indicates the operating temperature is 105 °C.
We have 2MB, 4MB, 8MB, 16MB and 32MB Flash memory variants as indicated in the specification Indicator Code. If there is an ‘R’ in the code, it suggests that the variant that we have has PSRAM. Mostly, we can see 2MB and ESP32 PSRAM 8mb variants. The last two characters are optional or left free for customisation.

Checking the Memory Capacities

Now, let's take a look at the Specification Identifier Codes of some real ESP32 Modules and try to identify the Memory Capacities of them.

LiteWing ESP32 S3 Modules

In the above image, the 1st module is the genuine ESP32-S3-WROOM-1 used in our LiteWing Drone. You can see that it has 8MB of Flash and no PSRAM. The second one is a module that is seen on most of the ESP32 S3 development boards. This particular model has 16MB of Flash and 8MB of PSRAM, denoted by N16R8. You may not be able to see these kinds of details in every module, as a lot of 3rd party manufacturers also manufacture these modules other than the original Espressif Systems.

Speaker Smart Glass Modules

In the above image, the 1st module is the genuine ESP32-S3-WROOM-1 used in our AI Voice Assistant Project. You can clearly see that it has 16MB of Flash and 8MB of PSRAM. We picked this ESP32 flash memory size because real-time audio processing requires a lot of computational power while also offering Wi-Fi and Bluetooth connectivity. The dual-core architecture is particularly valuable there, as one core handles network communication and system tasks, while the other focuses on audio processing and wake-word detection, ensuring smooth, responsive operation.

The second module in the above image is the ESP32-S3-MINI-1 used in our ESP32 AI Smart Glass Project. From its Specification Identifier Code, it's clear that it has 4MB of Flash and 2MB of PSRAM. We picked the dual-core architecture as we had to run the camera while uploading images and sending commands to the cloud all at the same time efficiently.

Conclusion

ESP32 Module Specification Identifier Code may look like some random letters and numbers, but they can give you useful information about the memory configuration. By learning how to read codes such as MCN8, MCN16R8, M0N4R2, and N16R8, one can quickly identify the Flash and PSRAM capacity of an ESP32 module without relying only on the product name or datasheet. This is a very helpful method for every Embedded Systems Engineer.

This article also teaches how to pick the right ESP32 variant based on the memory classification as well as the cores. We have mentioned how we picked the right ESP32 module for some of our projects as well. It's very clear that the next time you start an ESP32-based project, you’ll surely check and pick the right variant that suits your project requirements. 

Have any question related to this Article?

Top 80+ Beginner-Friendly EEE/ECE Mini-Projects for Electronics Worth Trying

Submitted by Merin on

For students in Electronics and Communication Engineering (ECE), a mini-project is more than just a submission; it’s an opportunity to turn concepts like sensors, microcontrollers, embedded systems, communication, automation, digital electronics, and IoT into a working prototype.
That’s where Circuit Digest can be a practical resource for electronics students. Our project resources are available for free, with GitHub code, circuit schematics, and detailed project walkthroughs, including mini projects that cover the basics of electronics. We help students explore a wide range of electronics and communication topics through academic projects, experimentation, and hands-on learning.
Whether you’re looking for beginner-friendly ECE/EEE mini projects for Electronics, an Arduino project, an embedded systems idea, an IoT prototype, a communication project, or a more advanced electronics concept, Circuit Digest can help you explore your options, narrow down your choices, and find a project that fits your interests and goals.

Learn by Building: Hands-On Electronics Mini Projects to Turn Ideas into Reality 

The best way to learn electronics is to build something yourself. Each project includes a working circuit, source code, and a simple explanation of how everything works together. So, instead of just following the steps, you get to understand what’s happening in the circuit and why it works. These projects give you a practical way to take what you’ve learned in the classroom and turn it into a working prototype that you can test, improve, and confidently demonstrate.

Automatic Toll Gate System Using Arduino

Automatic Toll Gate System Using Arduino

The “automatic toll gate system project” shows how simple sensors and a microcontroller can automate a real-life process like toll collection, making it an ideal project for anyone just starting in electronics. 

Dual Axis Solar Tracker System

Dual Axis Solar Tracker System

This dual-axis solar tracking system uses Arduino to move the solar panel horizontally and vertically based on the sun's location. This project, using LDR and servo motors, can increase energy output by up to 40% compared to fixed solar installations. 

Arduino Location Tracker

Arduino Location Tracker

This comprehensive project creates a fully functional GPS tracking system using Arduino UNO R3, SIM800L GSM module, and NEO-6M GPS module, a perfect low-cost DIY combination for vehicle monitoring, asset protection, or personal safety applications. 

Gas Leakage Detector

Gas Leakage Detector 

In this project, you'll learn how to build a simple and budget-friendly gas leakage detector using Arduino. This LPG gas leak detector uses an Arduino Uno and MQ-5 gas sensor to detect gas levels in the air and activates a buzzer and LED to alert users when a leak is present.

Smoke and Fire Alarm System

Smoke and Fire Alarm System 

In this project, you will also learn how to build an Arduino smoke alarm that sends SMS Alerts without needing to rely on the GSM module.  A fire and smoke alarm system using Arduino UNO R4 that sends real-time SMS notifications.

Send SMS with Arduino UNO R4 via Internet

Send SMS with Arduino UNO R4 via Internet 

In this tutorial, I’ll show you how to send SMS using Arduino UNO R4 and the free CircuitDigest Cloud SMS API. Whether it’s fire detection, motion sensing, or home automation, this setup has you covered.

Speed Sensor using Arduino

Speed Sensor using Arduino

In this article, we will learn how to calculate speed using Arduino and an IR sensor. By setting up the two IR sensors at a fixed distance from each other, we can track the time it takes for the object to travel between them. 

Detect the Direction of Sound

Detect the Direction of Sound

In this tutorial, let's learn how to find the direction of sound using Arduino and a few microphones. With the recorded time and the known distance between them, we can accurately calculate the object's speed using a formula.

RFID Door Lock System

 RFID Door Lock System

In this article, we will learn how to build an RFID door lock system using Arduino. It’s a fun and secure way to unlock the door. By integrating an RFID reader with an Arduino, this system will automatically open the door when an authorised RFID card or tag is scanned. 

Smart Home Using Arduino Uno R4 WiFi

Smart Home Using Arduino Uno R4 WiFi

Our Smart Home Using Arduino Uno R4 WiFi project is designed for home safety and convenience, integrating temperature, humidity, light, and gas monitoring. This project is perfect for anyone who wants to make their home or office a little smarter. 

Speaking Alarm Clock Using the XIAO ESP32-S3

 Speaking Alarm Clock Using the XIAO ESP32-S3

The ESP32 speaking alarm clock built in this tutorial replaces the beep at a scheduled time and expects the user to interpret the reason for the beep. This DIY Speaking Alarm Clock is developed using the XIAO ESP32-S3 microcontroller and a cloud-based Text-to-Speech (TTS) engine.

Noise Pollution Monitoring System

Noise Pollution Monitoring System

Noise Pollution Monitor tracks, analyses, and alerts on rising noise levels in real time. The noise pollution monitoring project continuously measures ambient sound and vibration, displays the readings locally, and pushes data to CircuitDigest Cloud to access the data from anywhere. 

Metal Detector System

Metal Detector System

This metal detector uses an affordable Wi-Fi-enabled microcontroller, a handful of available components, and a simple, hand-wound copper coil. We’ll guide you through winding the induction coil, assembling the pulse circuit, and flashing the code for this DIY PI metal detector.

Helmet Detection with Raspberry Pi

Helmet Detection with Raspberry Pi

This project is a compact, traffic monitoring device that uses a USB camera, Python, OpenCV, and the CircuitDigest Cloud API to automatically detect whether two-wheeler riders are wearing helmets in real time, without the need for on-device machine learning training or manual dataset labelling.

Raspberry Pi Waste Segregation System

Raspberry Pi Waste Segregation System

This project captures an image of waste using a USB camera and can be sent to an Image Processing API located in the Cloud for classification as either biodegradable or non-biodegradable. No trained machine learning model is required for classification, which is done using an API call. 

Raspberry Pi Parking Space Detection System

Raspberry Pi Parking Space Detection System

We have built a parking space detection system without any complex setup. We only need a Raspberry Pi board, a USB camera, and an account in CircuitDigest Cloud. The Requests library then sends the image to the CircuitDigest Cloud API using an HTTPS POST request with the API key.

T Flip-Flop

 T Flip-Flop

The flip-flop, aka latch, can also be understood as Bistable Multivibrator as two stable states. Generally, these latch circuits can be either active-high or active-low, and they can be triggered by HIGH or LOW signals, respectively.

NAND Gate with Transistors

NAND Gate with Transistors

In this article, we will go over how to build a NAND gate circuit with transistors. Transistors serve as the building blocks of logic gates, such as AND gates, NAND gates, OR gates, XOR gates, and other gates that are integral to integrated circuits. 

XOR Gate with Transistors

XOR Gate with Transistors

In this article, we will explore the inner workings of the XOR gate, including its truth table, logical symbol representation, circuit diagram, and practical construction using transistors. The XOR gate an essential component in various applications, from binary arithmetic to complex data encryption algorithms.

Panic Alarm Button Circuit

Panic Alarm Button Circuit

A Panic Alarm Circuit is used to send an emergency signal immediately to people in nearby locations to call for help or to alert them. The indication of an emergency can either be in the form of a visible or audible signal, which can be fixed a few meters away through wire.

Fire Alarm Circuit

Fire Alarm Circuit

Building a simple fire alarm system using a 555 Timer IC that will sense a fire (temperature rise in the surrounding area) and trigger the alarm. The key component of the circuit is a thermistor, which has been used as a fire detector or a fire sensor

Rain Alarm

Rain Alarm

A rain alarm is an application which detects rainwater and blows an alarm. They are useful device and plays an important role in various industries such as automobile, irrigation, and wireless communication. 

Fridge Door Alarm Circuit

Fridge Door Alarm Circuit

This circuit triggers the alarm if the door of the fridge is left open for a long time. When the door of the refrigerator is left open, the temperature inside the cabin will increase. This rise in temperature will be sensed by the thermostat, which will try to cool down the cabin.

Doorbell using IC 555

Doorbell using IC 555

The main feature of this doorbell is that we can control the time duration for which it keeps ringing upon pressing the switch. Also, we can control the oscillation frequency of the “doorbell sound” produced by the Doorbell (Here we are using a buzzer as a bell to illustrate).

Simple Flashing LED

Simple Flashing LED

An LED Flasher Circuit project is done with available electronic components and an easy-to-understand schematic. This tutorial will show you how to make an LED glow and fade at a certain interval

LED Dimmer Circuit

LED Dimmer Circuit

Building an LED ON and OFF circuit using a 555 timer IC and BC557 is very simple. In this circuit, the 555 timer IC is configured as an astable multivibrator, which means that it produces a continuous square wave output with a fixed frequency and duty cycle.

 DC-DC Boost Converter

DC-DC Boost Converter

In this article, we will learn about buck converters and design a very simple boost converter using a 555 timer and IRFZ44N, an N-channel MOSFET. A boost converter is a non-isolated type of switch-mode power supply that is used to step up the voltage.

 555 Timer-Based Buck Regulator

555 Timer-Based Buck Regulator

This circuit is basically a simple power electronics DC-DC Buck converter which can be used to step down voltage; its efficiency results in better battery life due to reduced heat generation, making it a lucrative option for smaller gadgets.

Positive and Negative Charge Pump Circuit

Positive and Negative Charge Pump Circuit

A charge pump is a type of circuit that is made out of diodes and capacitors configured in a specific configuration to get the output voltage higher than the input voltage or lower than the input voltage. By lower, I mean a negative voltage with respect to ground.

Simple Fading LED Light

 Simple Fading LED Light

The slow fade LED circuit is very simple; the 555 has been used in Astable mode, and a transistor is used to amplify the current. In Astable mode, the 555 IC oscillate at a particular frequency (depending on RC components), meaning the output at PIN 3 goes HIGH and LOW periodically.

Transistor Tester using 555 Timer IC

Transistor Tester using 555 Timer IC

In this tutorial, we will design a simple 555 TIMER-based circuit which will test the working of the transistor in seconds. This circuit is a convenient way to check the working of a transistor for newbies.

Audio Amplifier using 555 Timer IC

Audio Amplifier using 555 Timer IC

 In this tutorial, we are going to see how a 555 IC can be used as an audio amplifier. A low-power audio signal can be amplified using a 555 Timer IC.  We can test this circuit by blowing some air from the mouth towards the Mic; the speaker will generate sound.

Current Detector Circuit with 555 Timer

Current Detector Circuit with 555 Timer 

In this article, we build a simple current detector circuit with a 555 Timer and some passive components, which can help you to detect open live lines with ease. Before starting work on an electrical box and AC mains, one needs to verify that there is no AC leakage voltage.

Motion Detector Circuit using 555 Timer

Motion Detector Circuit using 555 Timer

In this tutorial, we are going to use an IR sensor with a NE555 Timer IC to detect motion and switch the AC load according to that. The 555 timer IC is used as a switch here. This circuit uses a digital timer IC; the operation of the circuit is fast and accurate, with even faster detection speeds. 

Simple LDR Circuit

Simple LDR Circuit

This dark detector circuit uses a 555 timer IC and an LDR (Light Dependent Resistor), which senses the light in the surroundings, and if it does not find light, it triggers the IC and turns on an LED attached to the circuit. 

Smart Dustbin Using Arduino

Smart Dustbin Using Arduino

This automatic smart dustbin is a decent gadget to make your home clean and attractive. Kids spread trash to a great extent with paper, wrappers, and numerous other things at home.  It opens automatically without touching to throw all trash and waste into this smart dustbin

Automatic Plant Watering System Using Arduino

Automatic Plant Watering System Using Arduino

This DIY automatic plant watering system project helps solve the common problem of forgetting to water plants while away from home. In this complete guide, we'll show you how to make an automatic plant watering system with soil moisture sensing and mobile app control capabilities.

ESP32 WLED Controller

ESP32 WLED Controller

 DIY tutorial is all about making our home lighting smart and fun without any complicated steps. We can connect a standard 12V LED strip to a tiny wi-fi chip and build our own ESP32 WLED controller.

Digital Keypad Security Door Lock

Digital Keypad Security Door Lock

In this project, I have built an Arduino Keypad Door Lock which can be mounted to any of your existing doors to secure them with a digital password. Of all the solutions, the low-cost one is to use a password- or PIN-based system.

Automatic Pet Feeder

Automatic Pet Feeder

Arduino automatic pet feeder that feeds your pet automatically at scheduled times. This project incorporates a DS3231 RTC (Real Time Clock) Module, allowing you to keep track of your pet's eating schedule and properly schedule feeding at specific times. 

3-Way Traffic Light Controller

3-Way Traffic Light Controller

This Arduino-based 3-Way Traffic Light Controller is a simple project which is useful for understanding how traffic lights work, which we see around us. It’s pretty simple and can be easily built on a breadboard 

Digital Thermometer Using Arduino

Digital Thermometer Using Arduino 

In this project, we have made an Arduino-based digital thermometer to display the current ambient temperature on a 16x2 LCD unit in real time. It can be deployed in houses, offices, industries, etc., to measure the temperature. 

Voice Controlled Home Automation

Voice Controlled Home Automation

 This model of a Modern Smart Home. These features demonstrate what a Smart Home would be like. It improves automation, control and monitoring of household devices and connects via standard means of communication for its operation.

Virtual Reality Interface with Gesture-Contro

Virtual Reality Interface with Gesture-Control 

This is a very interesting project in which we are going to learn how to implement virtual reality using Arduino and Processing. We will show you how you can simply wave your hand in front of a webcam and draw something on your computer. 

Arduino RFID Door Lock

Arduino RFID Door Lock

This RFID Door Lock can be made easily at home, and you can install it on any door. This door lock is just an electrically operated door lock which gets open when you apply some voltage (typically 12v) to it.

Automatic Street Light Controller

Automatic Street Light Controller

This Simple Automatic Street Light Circuit, using an LDR and a relay, will turn the light bulb on and off based on the lights in the surroundings. This circuit is quite simple and can be built with Transistors and an LDR; you don’t need any op-amp or 555 IC to trigger the AC load. 

Clap Switch

Clap Switch

A clap switch, it can be turned ON by any sound of approximately the same pitch as a clap. Here, we are using an Electric Condenser Mic for sensing the sound, a transistor to trigger the 555 timer IC, and a 555 IC to turn ON the LED through a low-voltage trigger. 

Handheld Arduino Game Console

Handheld Arduino Game Console 

This Arduino handheld game console is lightweight, easy to carry, and simple to build, making it perfect for hobbyists, electronics engineers, and young tech enthusiasts alike. It’s beginner-friendly, and it is enough to spark creativity and deeper learning.

BLE-based Proximity Control

BLE-based Proximity Control 

In this article, I am going to show you how to make a simple BLE presence detector with the help of an ESP32 and Arduino, and in the end, we will test these devices using BLE on my smartphone and a smartwatch. 

ESP32-Based Webserver

ESP32-Based Webserver

An ESP32-based web server is used to display the temperature and humidity values from the DHT11 sensor. ESP32 board will read the temperature and humidity data from the DHT11 sensor and display it on the Webpage. Here, IFTTT is also used to send email notifications when the temperature goes beyond a particular limit.

What Makes a Good Mini Project for ECE/EEE?

A good project has to fit your budget, available components, semester timeline, technical skill level, and, most importantly, give you something meaningful to explain during a viva or interview.
Finding an electronics and communication engineering mini-project topic is often the first challenge. Finding a project that matches your skill level, syllabus, budget and available components is the harder part.
Before choosing a topic, consider five things:

» Technical relevance- Does the project demonstrate something you have learned?
» Buildability - Can you obtain the components and complete the prototype within your deadline?
» Budget - Can you build it without spending unnecessarily on specialised hardware?
» Demonstration value - Can you clearly show the input, processing and output?
» Learning value - Will you be able to explain the circuit, code, limitations and future improvements?

These factors are particularly important for a low-cost electronics engineering mini project. Existing engineering project resources frequently emphasise that cost, time and resource availability strongly influence project selection.

Why use Circuit Digest for Mini Projects for Electronics?

  • Ready-to-use code: Many projects include source code and GitHub resources that students can study, modify, and experiment with.
  • Practical orientation: Focus on projects that can help connect classroom concepts with working electronics.
  • Technology exploration: Compare traditional circuit approaches with modern microcontroller, wireless and IoT-based implementations.
  • Learning before building: Understanding the circuit and technology behind a project makes it easier to troubleshoot and explain.
  • Useful for academic work: Project ideas can help students move from topic selection to prototyping, documentation and demonstration.

Circuit Digest helps make your journey easier by bringing electronics engineering mini project ideas and practical technical knowledge together in one place.

Explore Related Project Hubs: Artificial Intelligence | Electronics | IoT | Robotics | ESP32 | Raspberry Pi | Arduino Projects | Drone Projects |  Electronic Circuit 

Get Help Along the Way

Circuit Digest’s community is active and always ready to help. If you get stuck while building a project or simply want feedback on your work, you can connect with us and get guidance from our community. Join us on our WhatsAppand Instagram channelto ask questions, share your projects, and get the support you need. 

Have any question related to this Article?

iHub Robotics, the Kerala Startup Building Almost Every Layer of Its Humanoids In-House

Submitted by Staff on

Athil said his interest in robotics began during his school years, inspired partly by the Tamil film Enthiran and partly by Iron Man. He said he wanted to build his own version of that kind of technology. During that period, Arduino boards were difficult to get, so he started building small robots from whatever components were available.

ESP32 Video Player: SD Card to TFT Display

Playing smooth, full-motion video on low-power microcontrollers has been challenging due to memory and processing limitations. But with the dual-core speed, high SPI clock rates and an optimized decoding library on the ESP32, you can turn a simple development board into a working video player. If you want to create custom animations or add dynamic visual UI elements to your hardware projects, building this video player is a great way to push your hardware to its limits.
    In this tutorial, we are building an ESP32 video player that streams Motion JPEG (.mjpeg) files directly from a MicroSD card and displays them on a TFT display. We’ll walk you through all the hardware setup, video file formatting and code implementation needed to get your video running smoothly. You can also check out similar ESP32 Projects done previously here at Circuit Digest.

How Does This ESP32 Video Player Work?

Below is the block diagram of the ESP32 video player

1. SD Card

Stores the video as a Motion JPEG (.mjpeg) file—a simple sequence of individual JPEG images packed back-to-back.
The ESP32 reads this raw file stream continuously over the SPI bus.

2. RAM Buffer

Reads data in 4KB chunks into memory instead of byte-by-byte to prevent lag.
Scans incoming data to locate frame boundary markers and assemble complete JPEG images in RAM.

3. JPEG Decoder

Uses the lightweight JPEGDEC library to decompress the JPEG image directly in memory.
Converts the JPEG data into a 16-bit RGB565 pixel format that the screen can render.

4. TFT Display

Receives pixel data over a high-speed 40MHz SPI bus to update the ILI9341 screen.
Applies small timing delays to keep playback smooth and locked at a steady 15 FPS. Here is another Arduino touch screen calculator using a TFT LCD project where we showcased how to make a calculator with a TFT display and Arduino.

Converting Video to MJPEG Format:

To prepare your video file for the ESP32, you need to convert standard formats like .mp4 into a .mjpeg (Motion JPEG) file using a free web-based converter tool.
Paste the converted video file to the MicroSD card root directory. The file name should be video, and the format should be mjpeg. If we want to change the file name, we need to change the name in the code too to match the file name we choose.
The table shows the video conversion settings.

SettingsValue
Resolution320x240
File FormatMJPEG
FPS15
QualityMedium

Components Required

Below is the list of components required to build this project

S.NoComponentsSpecificationQuantity
1.MicrocontrollerESP32 Dev Module    1
2.TFT Display2.4 TFT SPI 240*320
(TJCTM24024-SPI)
    1

Circuit Diagram

 The following is the circuit diagram of the ESP32 video player

Connect the circuit as per the table below

TFT Display pinsESP32 Dev Module Pins
VCC3.3V
GNDGND
CSGPIO 2 
RSTGPIO 4
D/CGPIO 5
MOSIGPIO 23
SCKGPIO 18
LED3.3V
MISOGPIO 19
SD_CSGPIO 15
SD_MOSIGPIO 23
SD_MISOGPIO 19
SD_SCKGPIO 18

Coding

#include <SPI.h>
#include <SD.h>
#include <Adafruit_GFX.h>
#include <Adafruit_ILI9341.h>
#include <JPEGDEC.h>
// --- TFT Display Pins ---
#define TFT_CS   2
#define TFT_DC   5
#define TFT_RST  4
// --- SD Card Pin ---
#define SD_CS    15
// --- Frame buffer for one JPEG frame ---
// Increase if your frames are larger than this; watch ESP32 RAM limits.
#define FRAME_BUF_SIZE (80 * 1024)
static uint8_t *frameBuf = nullptr;
// --- Target playback rate ---
#define TARGET_FPS 15
#define FRAME_INTERVAL_MS (1000 / TARGET_FPS)
Adafruit_ILI9341 tft = Adafruit_ILI9341(TFT_CS, TFT_DC, TFT_RST);
JPEGDEC jpeg;
File videoFile;
// Drawing callback function
int JPEGDraw(JPEGDRAW *pDraw) {
 tft.drawRGBBitmap(pDraw->x, pDraw->y, pDraw->pPixels, pDraw->iWidth, pDraw->iHeight);
 return 1;
}
// Reads one JPEG frame (SOI 0xFFD8 ... EOI 0xFFD9) from videoFile into frameBuf.
// Returns frame size in bytes, or 0 if no more frames / error.
size_t readNextFrame() {
 // 1. Find SOI marker (0xFF 0xD8)
 int b1 = -1, b2 = -1;
 bool foundSOI = false;
 while (videoFile.available() >= 2) {
   b1 = videoFile.read();
   if (b1 == 0xFF) {
     b2 = videoFile.peek();
     if (b2 == 0xD8) {
       videoFile.read(); // consume the 0xD8
       foundSOI = true;
       break;
     }
   }
 }
 if (!foundSOI) return 0; // EOF reached without finding a new frame
 frameBuf[0] = 0xFF;
 frameBuf[1] = 0xD8;
 size_t idx = 2;
 // 2. Read bytes until EOI marker (0xFF 0xD9) is found
 int prevByte = 0;
 while (videoFile.available() && idx < FRAME_BUF_SIZE) {
   int curByte = videoFile.read();
   frameBuf[idx++] = (uint8_t)curByte;
   if (prevByte == 0xFF && curByte == 0xD9) {
     return idx; // complete frame captured
   }
   prevByte = curByte;
 }
 // Ran out of buffer space or file ended mid-frame
 Serial.print("WARN: frame incomplete or buffer too small, got ");
 Serial.print(idx);
 Serial.println(" bytes before running out of buffer/file");
 return 0;
}
void setup() {
 Serial.begin(115200);
 delay(1000);
 Serial.println("\n--- ESP32 Video Player Initializing ---");
 frameBuf = (uint8_t *)malloc(FRAME_BUF_SIZE);
 if (!frameBuf) {
   Serial.println("ERROR: Could not allocate frame buffer! Reduce FRAME_BUF_SIZE.");
   while (1) delay(1000);
 }
 Serial.print("Free heap after buffer alloc: ");
 Serial.println(ESP.getFreeHeap());
 tft.begin(27000000);
 tft.setRotation(1); // Landscape orientation
 tft.fillScreen(ILI9341_BLACK);
 Serial.println("Display Initialized.");
 if (!SD.begin(SD_CS)) {
   Serial.println("ERROR: SD Card initialization failed!");
   while (1) delay(1000);
 }
 Serial.println("SD Card initialized successfully!");
 if (!SD.exists("/video.mjpeg")) {
   Serial.println("ERROR: /video.mjpeg not found on SD card!");
   while (1) delay(1000);
 }
 videoFile = SD.open("/video.mjpeg", FILE_READ);
 if (!videoFile) {
   Serial.println("ERROR: Could not open /video.mjpeg!");
   while (1) delay(1000);
 }
 Serial.println("Found and opened /video.mjpeg on SD card!");
}
void loop() {
 unsigned long frameStart = millis();
 size_t frameSize = readNextFrame();
 if (frameSize == 0) {
   // End of file (or bad frame) — loop the video back to the start
   Serial.println("End of video, looping...");
   videoFile.seek(0);
   return;
 }
 if (jpeg.openRAM(frameBuf, frameSize, JPEGDraw)) {
   jpeg.decode(0, 0, 0);
   jpeg.close();
 } else {
   Serial.print("ERROR: jpeg.openRAM failed on this frame, size=");
   Serial.println(frameSize);
 }
 // Pace playback to target FPS
 unsigned long elapsed = millis() - frameStart;
 if (elapsed < FRAME_INTERVAL_MS) {
   delay(FRAME_INTERVAL_MS - elapsed);
 }
}
Have any question related to this Article?

Getting Started with the MAX32655 Feather Development Board

Submitted by Vishnu S on

Bluetooth Low Energy (BLE) has become one of the most widely adopted wireless technologies for battery-powered devices, offering reliable communication while consuming only a fraction of the power required by traditional wireless protocols. From wearable electronics and healthcare devices to wireless audio accessories and industrial sensors, BLE enables devices to remain connected for extended periods without sacrificing battery life.

Physical AI Tutorial: Train the SO-ARM101 Robot Arm with LeRobot and Jetson Orin NX

Submitted by Vedhathiri on

In this Physical AI tutorial, we will train a real robotic arm using Hugging Face LeRobot and imitation learning. The project uses the Seeed Studio SO-ARM101 leader–follower robotic arm and a reComputer J4012 powered by NVIDIA Jetson Orin NX.

We begin by configuring and calibrating the SO-ARM101, followed by teleoperation and dataset collection. We then record 150 human demonstrations, train an ACT policy using LeRobot and test whether the robotic arm can autonomously sort tomatoes and potatoes. This step-by-step LeRobot tutorial covers the complete workflow, including hardware setup, motor configuration, calibration, teleoperation, dataset recording, AI model training, troubleshooting and real-world evaluation. By the end, you will have everything needed to understand and reproduce the robotic learning pipeline used in this project. We have also worked on many projects related to robotics; you can check out our robotics projects for more ideas.

Both the SO-ARM101 robotic arm and the reComputer J4012 are Seeed Studio products. If you are planning to try a similar setup, you can use the links and promotional codes below: 

Check out SO-ARM101 Robotic Arm and use the promotional code KGYFKTZT to get $15 OFF

Check out reComputer J4012 and use the promotional code 90QQKB6Q to get 2% OFF.

Components Required for the Physical AI Robotic Arm Project

The following components are required to complete the robotic arm system. These include the mechanical, electronic, power, and communication components needed for operation. 

S.No                    ComponentQuantity                                            Purpose
1.Seeed Studio Leader Arm1Used by the human operator to teleoperate and demonstrate the required movements.
2.Seeed Studio Follower Arm1The actual working robotic arm that follows the leader arm and later runs the trained AI policy.
3.reComputer J4012-Edge AI Device with NVIDIA Jetson Orin NX1Main computing unit for training the AI model and running inference on the robot.
4.USB Camera1Captures visual observations of the workspace, objects, gripper, and bowls for vision-based learning and control.
5.12V Adapter2Powers the Follower arm motors and the Jetson Orin.
6.5V Adapter1Powers the Leader arm motors (7.4V / 5V motors).
7.Laptop1Used for initial setup, teleoperation, dataset recording, and development on Windows.
8.C-Type USB Cables2Connect the Leader arm, Follower arm, cameras, and Jetson to the Laptop.
9.Monitor, Mouse, and KeyboardOptionalProvides a display and input interface for the reComputer with Jetson Orin NX.

Essential Components for the project

Hardware and Software Understanding

This section provides an overview of the main hardware and software components used in the project, along with the reason for choosing them.

SO-101 Robotic Arm

The SO-101 is a low-cost, open-source 6-DOF robotic arm designed for research and learning applications. It consists of two arms: the Leader arm and the Follower arm. The Leader arm is operated by a human to demonstrate the required movements, while the Follower arm mimics those movements during teleoperation and later executes the trained AI policy independently. This Leader–Follower setup makes it easy to collect high-quality demonstration data for imitation learning.

Leader Arm

Unlike many industrial robotic arms, the SO-101 does not come with a separate external robot controller. Instead, it is supplied with two motor driver boards one for the Leader arm and one for the Follower arm and one for the Follower arm. These boards act mainly as a communication and power distribution interface between the computer and the motors.

Follower Arm

Motors: Feetech STS3215 Smart Servos

The SO-101 uses Feetech STS3215 smart bus servo motors. These are not ordinary DC motors. Each STS3215 motor has a built-in motor controller inside the servo itself. This internal controller handles position control, speed control, torque control, and feedback of the current joint position.

Smart Servo Motor

Because the intelligence is already inside the motor, the external driver board does not perform complex motion planning. It simply passes commands from the computer to the motors and supplies power. In simple terms, the motor contains the controller, the driver board acts as a communication bridge, and the computer (running LeRobot) sends the target positions.
Different gear ratios are used for different joints to balance torque and speed. The Leader arm typically uses lower-voltage (7.4V) motors so that it can be moved easily by hand, while the Follower arm is designed for actual task execution.

reComputer J4012 with NVIDIA Jetson Orin NX

The reComputer J4012 with NVIDIA Jetson Orin NX is a compact and powerful edge computing platform designed for AI workloads. In this project, it is used as the main system for training the imitation learning model and for running the trained policy on the robot.

reComputer J4012

Its GPU acceleration makes it suitable for training models such as ACT (Action Chunking with Transformers), which would be very slow or impractical on a normal laptop without a dedicated NVIDIA GPU.

Hugging Face and the LeRobot Framework

Hugging Face provides the ecosystem around LeRobot, including the LeRobot framework, model architectures, datasets and tools for sharing and managing robotics projects.

Hugging Face

In this project, the datasets and trained models were primarily stored locally during development, while the Hugging Face ecosystem provides the underlying framework and resources the project uses.

What is LeRobot?

LeRobot is an open-source robotics framework developed by Hugging Face. It provides a complete set of tools required for real-world robot learning, including hardware communication, teleoperation, dataset recording, model training, and policy evaluation. Without LeRobot, we would have to write most of this software pipeline from scratch.

Lerobot

Setting Up LeRobot on Windows

Installing Hugging Face LeRobot on Windows

In this project, we use the SO-ARM101 Low-Cost AI Arm Assembled Kit Pro (Leader + Follower) together with Hugging Face LeRobot. The goal of this phase on Windows is to complete motor setup and calibration, teleoperation, dataset recording, and dataset replay.
To begin the software setup, first install Miniconda. Visit the official Miniconda download page, download the Windows 64-bit installer, and run it. During installation, make sure to check the option “Add Miniconda3 to my PATH environment variable”. After the installation is complete, restart the computer. Next, open Command Prompt and create a dedicated environment for LeRobot by running the command below

conda --version
conda create -n lerobot python=3.10 -y
conda activate lerobot

If you get a Terms of Service error, run these commands first:

conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/msys2

After activating the environment, clone the official LeRobot repository and install it with Feetech motor support using the following commands:

git clone https://github.com/huggingface/lerobot.git
cd lerobot
pip install -e ".[feetech]"

Verify the Installation:

python -c "import lerobot; print(lerobot.__version__)"

A version number (such as 0.4.x or higher) should appear if the installation was successful.

Hardware Connection and Motor Setup

Before configuring the motors, it is recommended to clearly label them. Use F1 to F6 for the joints of the Follower arm and L1 to L6 for the Leader arm. The power supply must be connected carefully. The Leader arm always uses 7.4V motors and should be powered with a 5V adapter, while the Follower arm may use 12V motors depending on the configuration. Both the power supply and the USB cable must be connected, as the USB connection alone does not provide power to the motors.
Connect the 12V power adapter to the Follower arm driver board first, then connect the 5V power adapter to the Leader arm. After that, connect the USB cables from both driver boards to the laptop. To identify the correct COM ports for each arm, run the command below:

lerobot-find-port 

Then follow the on-screen instructions. Note down the port numbers for both the Follower and Leader arms.

Expected results:

Follower → COM(any number)
Leader → COM(any number)

Configuring SO-ARM101 Servo Motor IDs

Once the ports are identified, the motor IDs need to be configured. It is important to remove the daisy-chain connection before setting the ID of each motor; otherwise, errors may occur. Start with the leader arm and run the following command (replace the COM port with your actual port):

Setting up the Motor Id’s
lerobot-setup-motors --teleop.type=so101_leader --teleop.port=COM26

Follow the instructions carefully and connect one motor at a time. After completing the leader arm, repeat the same process for the follower arm using:

lerobot-setup-motors --robot.type=so101_follower --robot.port=COM23

Each motor must be assigned a unique ID from 1 to 6 so that the software can communicate with them individually.

Setting the Motor Id with Robotic Arm

Calibrating the SO-ARM101 Robot Arm

After setting the motor IDs, connect all the motors properly and proceed with calibration. When instructed, gently move each joint front and back and then move it to its minimum and maximum positions. You can also see the joint values changing on the screen. Then Press Enter to save the calibration values. First, calibrate the leader arm using the command:

lerobot-calibrate --teleop.type=so101_leader --teleop.port=COM26 --teleop.id=my_leader_arm
Calibrating Each Arm

Repeat the same process for the follower arm with the following command:

lerobot-calibrate --robot.type=so101_follower --robot.port=COM23 --robot.id=my_follower_arm

Calibration Working

Calibration Tips

  • Move joints slowly.
  • Do not force the mechanical limits.
  • Make sure the correct motor IDs are assigned.
  • Ensure the arm is properly powered.
  • If calibration produces unexpected values, stop and verify the motor ID and joint mapping.
  • Save the calibration only after verifying the joint movement.

Teleoperating the Robot Arm with LeRobot

Once both arms are calibrated, teleoperation can be tested. Run the following command:

lerobot-teleoperate --robot.type=so101_follower --robot.port=COM23 --robot.id=my_follower_arm --teleop.type=so101_leader --teleop.port=COM26 --teleop.id=my_leader_arm

When the Leader arm is moved, the Follower arm will follow the same movements in real time. Press Ctrl + C to stop teleoperation

Recording an Imitation Learning Dataset (Without Camera)

To record a dataset without using cameras, use the following command:

lerobot-record --robot.type=so101_follower --robot.port=COM23 --robot.id=my_follower_arm --teleop.type=so101_leader --teleop.port=COM26 --teleop.id=my_leader_arm --dataset.repo_id=semicon/arm_only_v1 --dataset.root="D:/lerobot_datasets/arm_only_v1" --dataset.num_episodes=3 --dataset.single_task="Pick and place small object" --dataset.push_to_hub=false --dataset.fps=30 --display_data=false

Make sure to change the COM port numbers and the dataset root path according to your system. This recording was performed without cameras.

Replaying a Recorded Dataset

To replay a recorded episode, use the command below:

lerobot-replay --robot.type=so101_follower --robot.port=COM23 --robot.id=my_follower_arm --dataset.repo_id=semicon/arm_only_v1 --dataset.root="D:/lerobot_datasets/arm_only_v1" --dataset.episode=0

This will make the Follower arm repeat the motion recorded in the selected episode.

Robotic Arm Playing Chess

Daily Startup Commands

Every time you start working, activate the environment and navigate to the LeRobot folder using:

conda activate lerobot
cd path\to\lerobot

After this, any of the above commands can be executed.

 Why Move to Jetson Orin NX for Training? 

Even though the robotic arm can perform basic operations such as teleoperation, recording, and replay on Windows, full model training is not practical on this platform. Dataset recording itself can still be done on Windows without major issues. However, training the AI model requires CUDA support for efficient GPU acceleration. If the computer does not have an NVIDIA GPU, or if it has limited RAM, the training process becomes extremely slow or may not run effectively at all. Considering these limitations, the model training process was moved to the reComputer J4012 with Jetson Orin NX. This is explained in more detail in the following section.

Setting Up LeRobot on NVIDIA Jetson Orin NX

Before starting the project on the reComputer J4012, it is important to understand a few key concepts related to AI model training and robot learning.

What is CUDA?

CUDA (Compute Unified Device Architecture) is a parallel computing platform developed by NVIDIA. It allows software to use the power of an NVIDIA GPU for general computing tasks, not just graphics.
A CPU usually processes tasks one after another, while a GPU can process thousands of smaller operations at the same time. CUDA is the technology that allows AI frameworks to use this GPU acceleration.

Cuda Definition

This is important because training a model like ACT involves a large number of mathematical calculations. These calculations are much faster on a GPU with CUDA. Since the Windows laptop used earlier did not have an NVIDIA GPU, training was impractical there. That is why the training process was moved to the reComputer J4012 with Jetson Orin NX

Different Types of Models Used in Robot Learning

In robot learning, different AI models can be used depending on the task:

  • ACT (Action Chunking with Transformers): This is the model used in this project. It learns from human demonstrations and predicts a sequence of robot actions. It is efficient and works well with smaller datasets.
  • Diffusion Policy: Generates actions step by step and is useful for complex or precise movements, but needs more computing power.
  • VLM (Vision-Language Model): Understands both images and text, useful when a robot must follow language instructions.
  • VLA (Vision-Language-Action Model): Combines vision, language, and actions for more general-purpose robot control. These models are larger and need more data.
    For this project, ACT was selected because it is suitable for imitation-learning tasks based on demonstration data and is practical for the relatively focused pick-and-place task used here.

Initial Setup on Jetson Orin NX

First, the Jetson Orin NX should be flashed with JetPack 6.1 or above for proper software compatibility and performance. To flash the Jetson Orin, follow the official document.
After that, the basic robot setup is done in the same way as on Windows:

  • Setting motor IDs
  • Calibration
  • Teleoperation
  • Record and replay testing

These steps follow the official Seeed Studio documentation. Checkcheck the document to get the full idea to setup in the linux. Once this basic setup is complete, the next stage is collecting data for the vegetable sorting task.

Dataset Collection and Training for Vegetable Sorting

To make the robotic arm learn the vegetable sorting task, we need to collect demonstration data. This data is recorded in the form of episodes.
What is an Episode?
An episode is one complete demonstration of the task.
For example:

  • Picking a tomato and placing it in the left bowl = 1 episode
  • Picking a potato and placing it in the right bowl = 1 episode
  • The AI model learns by studying many such examples. That is why multiple episodes are required.

Why around 120 episodes?
A single demonstration is not enough for the model to learn reliably. By collecting many episodes, the model sees different small variations in movement, timing, and object handling. In this project, around 120 episodes were collected in three groups:

  • Tomato only
  • Potato only
  • Both objects together

(picture of tomato, potato with the robotic arm)
This helps the model learn both individual object handling and combined sorting behaviour.

Recording the Vegetable Sorting Dataset

The following command is used to record the dataset using only the wrist camera:

Recording the dataset for the Tomato

conda activate lerobot
lerobot-record \
 --robot.type=so101_follower \
 --robot.port=/dev/ttyACM1 \
 --robot.id=my_follower_arm \
 --robot.cameras="{ wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
 --teleop.type=so101_leader \
 --teleop.port=/dev/ttyACM0 \
 --teleop.id=my_leader_arm \
 --dataset.repo_id=local/tomato_potato_wrist_fixed \
 --dataset.push_to_hub=false \
 --dataset.num_episodes=120 \
 --dataset.single_task="Sort the tomato into the left bowl and the potato into the right bowl" \
 --dataset.episode_time_s=45 \
 --dataset.reset_time_s=15 \
 --display_data=true

If you need to record all the episodes in one go, use the above command. Instead, if you need to collect the datasets in stages, like tomato only first, potato only, then both, follow the commands below:

Tomato only

conda activate lerobot
lerobot-record \
 --robot.type=so101_follower \
 --robot.port=/dev/ttyACM1 \
 --robot.id=my_follower_arm \
 --robot.cameras="{ wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
 --teleop.type=so101_leader \
 --teleop.port=/dev/ttyACM0 \
 --teleop.id=my_leader_arm \
 --dataset.repo_id=local/tomato_potato_wrist_fixed \
 --dataset.push_to_hub=false \
 --dataset.num_episodes=40 \
 --dataset.single_task="Sort the tomato into the left bowl and the potato into the right bowl" \
 --dataset.episode_time_s=45 \
 --dataset.reset_time_s=15 \
 --display_data=true

Recording the Dataset for each vegetable

Potato Only

lerobot-record \
 --robot.type=so101_follower \
 --robot.port=/dev/ttyACM1 \
 --robot.id=my_follower_arm \
 --robot.cameras="{ wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
 --teleop.type=so101_leader \
 --teleop.port=/dev/ttyACM0 \
 --teleop.id=my_leader_arm \
 --dataset.repo_id=local/tomato_potato_wrist_fixed \
 --dataset.push_to_hub=false \
 --dataset.num_episodes=40 \
 --dataset.single_task="Sort the tomato into the left bowl and the potato into the right bowl" \
 --dataset.episode_time_s=45 \
 --dataset.reset_time_s=15 \
 --display_data=true \
 --resume=true

Recording the dataset for the potato

Both Objects Together(placing tomato and potato at the same time)

lerobot-record \
 --robot.type=so101_follower \
 --robot.port=/dev/ttyACM1 \
 --robot.id=my_follower_arm \
 --robot.cameras="{ wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
 --teleop.type=so101_leader \
 --teleop.port=/dev/ttyACM0 \
 --teleop.id=my_leader_arm \
 --dataset.repo_id=local/tomato_potato_wrist_fixed \
 --dataset.push_to_hub=false \
 --dataset.num_episodes=40 \
 --dataset.single_task="Sort the tomato into the left bowl and the potato into the right bowl" \
 --dataset.episode_time_s=45 \
 --dataset.reset_time_s=15 \
 --display_data=true \
 --resume=true

Traning Potato

Training an ACT Policy with LeRobot:

After the dataset is collected, the ACT model is trained using the following command:

conda activate lerobot
lerobot-train \
 --dataset.repo_id=local/tomato_potato_wrist_fixed \
 --policy.type=act \
 --policy.repo_id=local/tomato_potato_wrist_fixed_act \
 --output_dir=outputs/train/tomato_potato_wrist_fixed_act \
 --job_name=tomato_potato_wrist_fixed_act \
 --policy.device=cuda \
 --batch_size=4 \
 --steps=100000 \
 --eval_freq=5000 \
 --save_freq=10000 \
 --log_freq=100 \
 --wandb.enable=false

Training the Model

Why these settings are used:

  • --policy.type=act → selects the ACT model
  • --policy.device=cuda → uses GPU acceleration
  • --steps=100000 → number of training steps
  • --save_freq=10000 → saves checkpoints regularly

If training is interrupted, it can be resumed from the last checkpoint:

conda activate lerobot
lerobot-train \
config_path=outputs/train/tomato_potato_wrist_fixed_act/checkpoints/last/pretrained_model/train_config.json \
 --resume=true

Testing the Physical AI Robot Arm

After training, the model is tested on the real robot using:


conda activate lerobot
lerobot-record \
 --robot.type=so101_follower \
 --robot.port=/dev/ttyACM1 \
 --robot.id=my_follower_arm \
 --robot.cameras="{ wrist: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30} }" \
 --dataset.repo_id=local/eval_wrist_fixed \
 --dataset.push_to_hub=false \
 --dataset.num_episodes=1 \
 --dataset.single_task="Sort the tomato into the left bowl and the potato into the right bowl" \
 --dataset.episode_time_s=600 \
 --dataset.reset_time_s=15 \
 --display_data=true \
 --policy.path=outputs/train/tomato_potato_wrist_fixed_act/checkpoints/last/pretrained_model

In this step, the robot is no longer controlled by the leader arm. Instead, the trained model controls the follower arm based on the camera input and the learned behaviour. You can check out the model by placing the tomato and potato one by one.

Output of Sorting

Common SO-ARM101 and LeRobot Errors and Fixes

This section provides solutions to the most common problems encountered during the project, along with their possible causes and recommended fixes. Use it as a quick reference whenever an error occurs during setup, operation, or training.

S.NoProblemCauseSolution
1.Dataset already exists error while starting evaluationLeRobot does not overwrite an existing dataset folder by default. A previous evaluation dataset with the same name was already present.Deleted the existing dataset folder using rm -rf and then restarted the evaluation command.
2.Cameras disconnected during evaluationUSB bandwidth issues or an unstable camera connection caused the system to lose access to the camera devices mid-run.Reconnected the cameras, verified the indexes using lerobot-find-cameras opencv, and restarted the evaluation.
3.The training process got interrupted before completionLong training duration and system interruption stopped the process around intermediate checkpointsResumed training from the last saved checkpoint using the --resume=true option instead of restarting from zero.
4.Arm did not return to home position after evaluation episodesDuring policy evaluation without the leader arm, the reset phase has no active control input, so the arm stays in its final positionManually move the arm back to the starting/home position before each new episode
5.Magnitude/range error during calibrationJoint was not moved through its full range, or the motor moved too little / too much during calibrationRecalibrate the arm carefully by moving each joint slowly from minimum to maximum position as instructed
6.Motor ID setup failed / communication errorMotors were still connected in daisy-chain mode while assigning IDsDisconnect the daisy-chain, connect one motor at a time, set the ID, then reconnect

Future Improvements

While the current system successfully demonstrates imitation learning for tomato and potato sorting, there is still significant room for improvement. The present setup mainly relies on a wrist camera with fixed object positions, so introducing an additional external camera from the top or side can provide better visual understanding and help the robot handle objects placed in different locations more reliably. Beyond laboratory experiments, the same learning pipeline can be adapted for practical applications such as warehouse sorting, educational robotics platforms, and simple industrial pick-and-place tasks. The system can also be expanded to handle a wider range of objects by training it to sort multiple items like different fruits or colored blocks
Adding force or tactile sensing to the gripper would further improve grasp stability, especially when dealing with soft or irregular objects. In the longer term, mounting the arm on a mobile base could allow the robot to perform sorting tasks across different locations rather than remaining restricted to a fixed table setup, making the overall system more flexible and closer to real-world use.

 Conclusion

This project showed how a low-cost robotic arm can learn a real task by watching human demonstrations. Using the SO-101 arm and LeRobot, we completed the full process from hardware setup and teleoperation to data recording, model training, and final testing. The robot was trained to sort a tomato and a potato into different bowls. During the project, we worked on both Windows and the Jetson Orin NX, and this helped us understand which steps are easy on a normal laptop and which steps need stronger computing power.

We also faced practical problems such as camera disconnection, training interruptions, and limited performance when both objects were present. By solving these issues step by step, the system became more stable and useful. Overall, this project gives a clear and simple example of how imitation learning can be used to build practical robotic systems with affordable hardware, and it can be extended further for more complex tasks in the future.

Have any question related to this Article?

Component Selection Guide for AI Hardware: What to Check Before Finalizing MLCC, MCU, and CPU/GPU Peripheral BOMs

From June to July 2026, AI hardware demand affected more than just GPUs, CPUs, and high-bandwidth memory. TrendForce reported that DRAM contract prices would rise by 13%–18% in Q3 2026. NAND Flash contract prices would rise by 10%–15%. The main drivers are AI inference systems and large data center deployments.

The pressure also moved into passive components. Recent market reports show that ordinary MLCC spot prices increased by 15%–20% since late February. High-capacitance MLCCs used in AI servers, including 10µF and 22µF parts, saw average price increases of 50%–60%.

This changes the selection process for engineers and procurement teams. A component is not suitable just because its electrical parameters match the schematic. Before finalizing the BOM, teams also need to check availability, second-source options, validation effort, traceability, and long-term supply risk.

1. MLCC Selection: Look Beyond Capacitance and Package Size

Take Samsung Electro-Mechanics CL10A226MP8NUN# / CL10A226MP8NUNE as an example. This MLCC series is rated at 22µF, 10V, X5R, in a 0603 package. Engineers often use it for power decoupling, compact DC-DC output filtering, communication modules, and dense PCB layouts. But in AI edge devices, server accelerator boards, and industrial control modules, this type of high-capacitance, small-case MLCC should not be selected only by matching “22µF, 10V, 0603.” Engineers should check the following points:

  • Whether the effective capacitance after DC bias is still enough for rail stability
  • Whether the 10V rating leaves enough derating margin
  • Whether X5R or X7R temperature behaviour fits the operating environment
  • Whether 0603 can be changed to 0805 if supply becomes tight
  • Whether the application requires AEC-Q200 qualification for automotive or high-reliability industrial use

In June, Astute Group reported that demand from AI infrastructure is tightening global MLCC supply. Some lead times now exceed 20 weeks. These constraints are expected to continue into 2027. This means high-capacitance MLCCs should have approved alternatives early in the design stage. Teams should not wait until production demand has already started.

2. MCU Selection: Same Core Does Not Mean Drop-In Replacement

Take the STM32F103C8T6 as a common example for MCU-based control boards. According to STMicroelectronics, this MCU uses an Arm Cortex-M3 core. Its CPU runs at up to 72MHz. It has 64KB of Flash memory. Engineers often use it in motor control, USB, CAN, and general industrial control applications.

MCUs may face long lead times or price pressure. In that situation, it is risky to pick an alternative just because it has the same Arm Cortex-M3 core or a similar package. Engineers need to check several details. They should verify pin definitions and pin multiplexing functions. They should also check ADC channels, PWM outputs, CAN and USB interfaces, and external crystal pins. Flash and RAM capacity, boot mode, debug interface, and software library compatibility also matter.

Temperature grade is just as important. A commercial-grade MCU may work for a consumer device. But industrial and automotive designs often need a wider temperature range. They may also require AEC-Q100-related validation. For mature products, replacing an MCU is not a simple purchasing step. It may involve a schematic.

3. CPU/GPU Peripheral Components: Power and Interconnect Matter

CPUs and GPUs get the most attention in AI platforms. But the system's overall performance depends on many other components. NVIDIA's GB200 NVL72 is an example. It connects 36 Grace CPUs and 72 Blackwell GPUs. This system uses a rack-scale design with liquid cooling. Such a large platform needs more supporting parts. These include PMICs, MOSFETs, inductors, high-capacitance MLCCs, high-speed connectors, memory devices, PCB materials, thermal sensors, and cooling-control components.

When selecting CPU/GPU peripheral components, teams should check the following points:

  • Whether the PMIC or voltage regulator has a qualified second source
  • Whether MOSFETs meet voltage, current, RDS(on), switching loss, and thermal requirements
  • Whether inductors meet saturation current and temperature-rise limits
  • Whether high-speed connectors meet signal integrity, insertion loss, temperature rise, and mating-cycle requirements
  • Whether DRAM, NAND, or related memory devices are exposed to current price and allocation cycles
  •  Whether fan control, liquid-cooling control, or temperature‑monitoring components have long lead-time risk

One important point to note is that a project can still be delayed even when the main CPU or GPU is secured. A small power, connector, sensing, or memory component can become the actual delivery bottleneck if it has no approved alternative.

4. A Practical Selection Workflow for Engineering and Procurement Teams

Step 1: Re-rank the BOM by supply risk, not only by unit price. 
Low-cost MLCCs, small MCUs, connectors, or power components can become production blockers if they are hard to replace. AI hardware BOMs should mark MLCCs, MCUs, PMICs, memory devices, connectors, MOSFETs, inductors, and CPU/GPU power-rail components as priority review items.

Step 2: Build one shared alternative-part validation table.
The technical side should include package, pinout, electrical ratings, temperature grade, certification, lifecycle status, and software compatibility. The procurement side should include stock availability, lead time, date code, packaging condition, price validity, and manufacturer traceability. A part should move into validation only when both sides are acceptable.

Step 3: Move sourcing checks into the design stage.
ECIA’s June Industry Pulse report tracks sales expectations, product cancellations, product decommits, and component lead times across major categories and end markets. This shows why sourcing is now part of engineering decision-making. It is not only a purchasing task after design release.

Teams can use manufacturer data during design, pilot production, and mass production. They can also use official suppliers, other distributors, and the global spot market. They check if key parts are in stock. They check if these parts are traceable. They check if they have reliable replacements. In this workflow, WIN SOURCE can assist with BOM execution. We help procurement teams review supply availability. We help them review alternative options. We help them verify traceability. We help them assess delivery risks. We cover many component types. These include ICs, MLCCs, MCUs, memory chips, power devices, and connectors. Our support is especially useful for parts with long lead times. It also helps with parts in short supply. It helps with obsolete parts. It also helps with small-batch validation.

AI demand is changing how engineers and procurement teams select components. Parameter matching remains the foundation of component selection, but it is not sufficient to support a complete BOM decision. For MLCCs, MCUs, PMICs, memory devices, and CPU/GPU peripheral components, a more reliable selection process should include technical compatibility, supply stability, alternative validation time, and quality traceability before finalising the BOM.

 

 

© 2026 Win Source Electronics. All rights reserved. This content is protected by copyright and may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Win Source Electronics.

Have any question related to this Article?