Speech Recognition Based Beanbot - Beach Cleaning Robot

BEANBOT is a speech recognition based Beach Cleaning Robot, Which will clean the Beach by separating Hard Objects, waste materials from sand. The Principle of Beach cleaning robot is sand Sieving Mechanism. It Sieves the Sand and Collects Wastes into the Bin.

When Robot is powered on, it will configure ESP32 Wifi and it will automatically connect to Wifi network configured in the code. Once it is done, The ST7789 Display board Connected to Maixduino will display the "Welcome" and it waits for the Command.
The Command " Hello Beanbot" or "Hey Beanbot" Wakes up the Robot and replies back with "Hello Angu" (Angu -My Name). Then it will Wait for the Next Command.

The Next Command is "Engines On" it will Configure some IOs and replies back "Engines Activated" and waits for next Command, if the Robot does not detect the Voice Command or it will be reply "Speak again".
The Next Command is "START", After that, it will Rear Motor to move forward for 1secs, sand collected in the Sieve and Rear motor Stops and it activates Solenoid sieve for 10secs, The Sieve Moves axially, During the operation Sand is Fell down on ground and waste materials vibrated to Waste bin present at the center of robot, After robot Continues to move forward. if any Obstacle is sensed, the Robot will stop and turn towards right and start moving

Impact Statement

India has more than 2000+ Beaches, It is easily Polluted by people. This Causes Harmful Effects to Beach environment and also sometimes to marine environment. To avoid and to save environment, it is necessary to clean the beaches periodically. But, Beaches are very Big area, Hence it requires huge manpower to maintain it every time. i propose Beach cleaning Robot makes Great impact in Cleaning the Beaches periodically

The collected Waste Materials put into bin which is present in the robot. It also Sends Robot Live information to IOT Cloud (Thingspeak.com) a Mathworks platform. It also has an ultrasonic sensor which transmits Ultrasonic waves and receives back to avoid obstacle

Components Required

1. Sipeed Maixduino Board - 1
2. ST7789 Video Display - 1
3. 3S 18650 Batteries (3x 4.1V batteries) -1
4. 12V Battery Management System - 1
5. USB-C to 12V Battery Booster Charging Module - 1
6. Ultrasonic Sensor - 1
7. 12V 100rpm Geared Motor - 2
8. BO Wheels - 2
9. 12V Solenoid - 1
10. 3A Fuse - 1
11. Terminal Connectors -2
12. Soleniod Drive Circuit - 1
13. Jumpers wires
14. DCDC Converter Module (12V to 9V for Maixduino)
15. 12V Motor Driver module - 1
16. SPDT Switch - 1
17. MOSFET IRF540N

To see the full demonstration video, click on the YouTube Video below.

Circuit Diagram Explanation

• Next, let's look at the connections used in this project. The circuit diagram is straightforward.

• I used Fritzing to create the circuit diagram. I used 18650 Lithium Ion Battery 4.1V Connected in Series for Power supply. 3 Li-ion Batteries are Connected in Series in order to provide 12V Power supply.

• The +12V is Connected to BMS (Battery management System) to manage Batteries in Uneven Discharging and Short Circuit Protection, After the BMS, the Supply is Connected to USB-C Battery Charger Via SPDT Switch and another End of Switch is connected to Load, where we can switch between Load and Charger

• In the Load side, Series 3A Fuse is Connected for safety Purpose, In order to prevent Current >3A in Load side. After Fuse, +12V is Distributed to Motor Driver module, DC-DC Converter and Solenoid Drive control circuit

• The Two Motors with 200rpm 1.5Kg-cm Torque are connected to motor Driver modules L293D independently

• The output of DC-DC Converter is connected to Maixduino DC Jack supply.

• TFT Display is connected to Maixduino Board via Flex PCB Connection. Maixduino Digital Pins 2,3,4,5 are connected to Motor A+, A-, B+, B- Pin 6 &10 is Connected to Motorenable1 &2 and Pins 7 & 8 are Connected to Trig pin and Echo pin of the Ultrasonic sensor. Pin 9 is Connected to Gate pin of MOSFET IRF540N. Drain is connected to Solenoid+ Wire and in series with +12V. A Diode to be connected across the Solenoid with Anode towards +12V. Diode is used to reduce the Voltage Spikes during Solenoid Discharge.

• For +5V, Supply is taken from Maixduino Board to ultrasonic Sensor

Design Calculations

Robot is Completely Designed & Developed from scratch with necessary calculations
Please find the Following Calculations in the Link :

https://docs.google.com/spreadsheets/d/1DIYAKPFuDpjvL-CKQGZks4cxsgKYU702xkRURFhcByw/edit?usp=sharing

1. Robot Torque Calculation
2. Power analysis (Power consumption Calculation)
3. Battery Calculation & Run time Calculation

CAD Parts Design

All Mechanical Parts are crafted on paper and designed, Simulated in Fusion 360 and then 3D Printed with Precision of 0.1mm.

Some of the parts are shown below :

Sieve Support CAD Design-

Rear Wheel Support CAD Design-

Complete Mechanical Design 

 

Code Explanation:

Part 1 : Initialisation

(Includes and Defines: Includes necessary libraries and defines pin numbers for sensors and Motors.
Global Variables: Declares global variables for the Display Module, Motor Pins and Solenoid
Setup Function: Initializes serial communication, configures pin modes, attaches Motors to pins, initializes the display board, and connects to WIFI Network through Communication with ESP32.)

#define motora 2 // Motor a+ pin
#define motorb 3 // Motor a- pin
#define motorc 4 // Motor b+ pin
#define motord 5 // Motor b- pin
#define motorenable1 6 // Speed control for Motor
#define trigPin 7 // Trigger pin of the sensor
#define echoPin 8 // Echo pin of the sensor
#define solenoid 9 // Soleniod Control pin
#define motorenable2 10 // Speed control for Motor

const int sensorPin = A0; // Sensitivity in mV per Ampere (adjust based on your sensor's rating)
const float sensitivity = 185.0; // for ACS712-05B (185 mV/A)
const int baselineOffset = 2500; // Sensor baseline offset in millivolts (typically 2500 for ACS712 sensors with a 5V supply)
const int numSamples = 100; // Number of samples for averaging

#include <Sipeed_ST7789.h>
#include "Maix_Speech_Recognition.h"
#include "voice_model_repo.h"
#include "WiFiEsp.h"

#ifndef HAVE_HWSERIAL1
#include "SoftwareSerial.h"
SoftwareSerial Serial1(6, 7); // RX, TX
#endif

char ssid[] = "Angu"; // your network SSID (name)
char pass[] = "naannallavan"; // your network password
int status = WL_IDLE_STATUS; // the Wifi radio's status

const char* apiKey = "X903CMCOLAM1X61S"; // Replace with your ThingSpeak Write API Key
const char* server = "api.thingspeak.com"; // ThingSpeak server
WiFiEspClient client;

long duration;
int activate = 1;
int distance;
SPIClass spi_(SPI0); // MUST be SPI0 for Maix series on board LCD
Sipeed_ST7789 lcd(320, 240, spi_, SIPEED_ST7789_DCX_PIN, SIPEED_ST7789_RST_PIN, DMAC_CHANNEL2);

SpeechRecognizer rec;
void Clean_activate();
void setup()
{

Serial.begin(115200); // initialize serial for debugging
Serial1.begin(115200); // initialize serial for ESP module
WiFi.init(&Serial1); // initialize ESP module
// attempt to connect to WiFi network
while ( status != WL_CONNECTED)
{
Serial.print("Attempting to connect to WPA SSID: ");
Serial.println(ssid); // Connect to WPA/WPA2 network
status = WiFi.begin(ssid, pass);
}

Serial.println("You're connected to the network");

pinMode(motora, OUTPUT);
pinMode(motorb, OUTPUT);
pinMode(motorc, OUTPUT);
pinMode(motord, OUTPUT);
pinMode(trigPin, OUTPUT);
pinMode(echoPin, INPUT);
pinMode(solenoid, OUTPUT);
digitalWrite(motora, LOW);
digitalWrite(motorb, LOW);
digitalWrite(motorc, LOW);
digitalWrite(motord, LOW);
digitalWrite(solenoid, LOW);
digitalWrite(motorenable1, LOW);
digitalWrite(motorenable2, LOW);

lcd.begin(15000000, COLOR_RED);

rec.begin();
Serial.println("init model...");
rec.addVoiceModel(0, 0, beanbot_0, hello_beanbot_0);
rec.addVoiceModel(0, 1, beanbot_1, hello_beanbot_1);
rec.addVoiceModel(0, 2, beanbot_2, hello_beanbot_2);
rec.addVoiceModel(0, 3, beanbot_3, hello_beanbot_3);
rec.addVoiceModel(1, 0, on_0, Engines_on_0);
rec.addVoiceModel(1, 1, on_1, Engines_on_1);
rec.addVoiceModel(1, 2, on_2, Engines_on_2);
rec.addVoiceModel(1, 3, on_3, Engines_on_3);
rec.addVoiceModel(2, 0, start_0, engine_start_0);
rec.addVoiceModel(2, 1, start_1, engine_start_1);
rec.addVoiceModel(2, 2, start_2, engine_start_2);
rec.addVoiceModel(2, 3, start_3, engine_start_3);
rec.addVoiceModel(3, 0, stop_0, engine_stop_0);
rec.addVoiceModel(3, 1, stop_1, engine_stop_1);
rec.addVoiceModel(3, 2, stop_2, engine_stop_2);
rec.addVoiceModel(3, 3, stop_3, engine_stop_3);
Serial.println("init model ok!");
lcd.fillScreen(COLOR_RED);
lcd.setTextSize(2);
lcd.setTextColor(COLOR_WHITE);
lcd.setCursor(100,100);
lcd.setRotation(3);
lcd.fillScreen(COLOR_RED);
lcd.setCursor(100,100);

}

Part 2 : Main Loop

(The Main loop Consists of Various inner loops which analyses the Speech Reconized with Speech files and it will Prints Various States.
It also does the Cleaning mechanism)

void loop()
{
int res;
while(1)
{
delayMicroseconds(200);
lcd.fillScreen(COLOR_RED);
lcd.setCursor(50,100);
lcd.print("Welcome");
res = rec.recognize();
Serial.printf("res : %d ", res);
if (res == 1)
{
delayMicroseconds(200);
lcd.fillScreen(COLOR_RED);
lcd.setCursor(100,100);
lcd.print("Hey Angu");
break;
}
else
{
delayMicroseconds(50);
lcd.fillScreen(COLOR_RED);
lcd.setCursor(50,100);
lcd.println("Speak again");
delay(500);
}
}
while(1)
{
delayMicroseconds(200);
res = rec.recognize();
Serial.printf("res : %d ", res);
if(res == 2)
{
lcd.fillScreen(COLOR_RED);
lcd.setCursor(5,100);
lcd.print("Engines activated");

delay(2000);
break;
}
else
{
lcd.fillScreen(COLOR_RED);
lcd.setCursor(50,100);
lcd.print("Speak again");
delay(500);
}
}
while(1)
{
delayMicroseconds(50);
res = rec.recognize();
Serial.printf("res : %d ", res);
if(res == 3)
{
delayMicroseconds(200);
lcd.fillScreen(COLOR_RED);
lcd.setCursor(5,100);
lcd.print("Beach Cleaning Started");
break;
}
else
{
lcd.fillScreen(COLOR_RED);
lcd.setCursor(100,100);
lcd.print("Speak again");
delay(500);
}
}
do
{

int engines = 1;
Clean_activate();

printCurrentNet();
printWifiData();
sendDataToThingSpeak(activate, engines);
}while(1);
}

Part 3: User Functions - Ultrasonic Sensor, Solenoid and Motor activation

void Clean_activate()
{
analogWrite(motorenable1,200);
analogWrite(motorenable2,200);
Serial.println("motorenable");
digitalWrite(trigPin, LOW);
delayMicroseconds(2);

// Trigger the ultrasonic pulse by setting the trigPin HIGH for 10 microseconds
digitalWrite(trigPin, HIGH);
delayMicroseconds(10);
digitalWrite(trigPin, LOW);

// Read the echoPin to measure the duration of the pulse
duration = pulseIn(echoPin, HIGH);

// Calculate the distance in centimeters
distance = duration * 0.034 / 2; // Speed of sound is approximately 0.034 cm/µs

// Display the distance in the serial monitor
Serial.print("Distance: ");
Serial.print(distance);
Serial.println(" cm");

// Wait for a short moment before taking another measurement
delay(200);
if (distance > 15)
{
lcd.fillScreen(COLOR_RED);
lcd.setCursor(10,100);
lcd.print("Motor forward");
motorforward();
delay(500);
motorstop();
delay(1000);
lcd.fillScreen(COLOR_RED);
lcd.setCursor(10,100);
lcd.print("Sand Seiving");
solenoidcontrol();

}
else
{
lcd.fillScreen(COLOR_RED);
lcd.setCursor(10,100);
lcd.print("Obstacle detected");
motorstop();
delay(2000);
motorreverse();
delay(2000);
motorright();
delay(2000);
}
}
void solenoidcontrol()
{
for(int i=0; i<20;i++)
{
digitalWrite(solenoid,HIGH);
delay(200);
digitalWrite(solenoid,LOW);
delay(200);
}

}
void motorforward(){
digitalWrite(motora, HIGH);
digitalWrite(motorb, LOW);
digitalWrite(motorc, HIGH);
digitalWrite(motord, LOW);
Serial.println("Motorforward");
}

void motorreverse(){
digitalWrite(motora, LOW);
digitalWrite(motorb, HIGH);
digitalWrite(motorc, LOW);
digitalWrite(motord, HIGH);
Serial.println("Motorreverse");
}

void motorright(){
digitalWrite(motora, LOW);
digitalWrite(motorb, HIGH);
digitalWrite(motorc, HIGH);
digitalWrite(motord, LOW);
Serial.println("Motorright");
}
void motorleft()
{
digitalWrite(motora, HIGH);
digitalWrite(motorb, LOW);
digitalWrite(motorc, LOW);
digitalWrite(motord, HIGH);
Serial.println("Motorleft");
}
void motorstop()
{
digitalWrite(motora, LOW);
digitalWrite(motorb, LOW);
digitalWrite(motorc, LOW);
digitalWrite(motord, LOW);
Serial.println("Motorstop");
}
void printWifiData()
{
// print your WiFi shield's IP address
IPAddress ip = WiFi.localIP();
Serial.print("IP Address: ");
Serial.println(ip);

}

void printCurrentNet()
{
// print the SSID of the network you're attached to
Serial.print("SSID: ");
Serial.println(WiFi.SSID());
}

Part 4: IoT - Data Sending to cloud

void sendDataToThingSpeak(int activate, int engines) {
if (client.connect(server, 80)) { // Connect to ThingSpeak on port 80
Serial.println("Connected to ThingSpeak");
// Create the URL for the HTTP request
String url = "/update?api_key=" + String(apiKey) + "&field1=" + String(activate)+ "&field2=" + String(engines);
// Send HTTP GET request to ThingSpeak
client.println("GET " + url + " HTTP/1.1");
client.println("Host: " + String(server));
client.println("Connection: close");
client.println(); // End of headers

// Wait for the response from the server
long timeout = millis() + 5000; // 5 seconds timeout
while (client.connected() && millis() < timeout) {
if (client.available()) {
String response = client.readString();
Serial.println("Server Response: ");
Serial.println(response); // Print the server response
break;
}
}

if (millis() >= timeout) {
Serial.println("Server response timed out.");
}

client.stop(); // Close the connection
Serial.println("Data sent to ThingSpeak.");
} else {
Serial.println("Connection to ThingSpeak failed.");
}
}

Working Demonstration:

Below, you can see the working image of the project. The Robot is collecting Sand and Sieving the sand and collects the Wastes in the bin

IOT Data:

Thingspeak.mathworks.com

For all the code, follow the GitHub link below: