1. Home
  2. Microcontroller Based Projects
  3. Mask Detection System using ESP32 Cam and M5Stack Core2

Mask Detection System using ESP32 Cam and M5Stack Core2

Published  February 25, 2022   0
A Aniket Dhole
Author
Mask and Body Temperature Checker System

In this project, we will make a device to detect if a person is wearing a mask and has a healthy heart rate using M5Stack Core2, ESP32-Cam, AWS IoT, and Edge Impulse. After the pandemic, most of the offices will start for on-site work and schools will start, during this time we also need to take care of any employee or student is having correct body temperature and is wearing a mask. Instead of hiring someone and risking their life to check the temperature and mask for so many employees and students, is a tough task. So to solve this problem, we want to create a device that monitors their heart rate and checks wearing masks using the Edge Impulse model on AWS IoT Edukit and Embedded Camera. 

Previously, we have used Heart Rate Sensor in many projects, you can check them out

Also learn more about M5Stack Core2 by going through Introducing M5Stack Core2 - An ESP32 based IoT Development Kit.

Component Required for Mask and Health Management System

Project Used Hardware

  • ESP32 CAM,
  • M5Stack Core2,
  • KY-039 Heart Rate Sensor,
  • IR Distance Sensor,
  • Perf Board and Wires

Project Used Software

  • Arduino IDE,
  • Edge Impulse,
  • AWS IoT

Project Hardware Software Selection

All the Computer Vision will be completely done over ESP32 Cam using a Deep Learning model. The Mask Detection model is trained on Edge Impulse and the generated library is used in ESP32 CAM. The Data is sent to AWS IoT using MQTT and major alerts like Person is Not Wearing Mask is sent as an email to Admin using AWS SES Flow:

  • The IR Distance sensor will check when the user is standing in front of the Device and start the camera and heart sensor
  • Read the Image and inference it on Mask Model.
  • Check whether Employee/Person is wearing Mask using Mask Classification Model trained on Edge Impulse
  • Send Predictions to M5Core2 using UART communication.
  • Then Check the Pulse Rate using Pulse Sensor and send it to M5Core2 to display on the screen.
  • If results are showing that the person is not healthy then that case will be reported to the admin using AWS IoT and SMTP via SES.

AWS IoT and SES

Whenever the user is detected the predictions are sent to AWS via MQTT to AWS IoT Cloud and when there are any people who are not wearing masks then it will also be send via eMAIL using SMTP of AWS SES service.

AWS IoT

  1. Initially Create Thing in AWS IoT and set up the certificate and MQTT URL.
  2. It will generate a unique URL for the host URL and also give a topic in form of "$aws/things/NAME/FUNCTION"
  3. Set Up the Credentials in code as: int PORT = 8883; char MQTT_SUB[] = "$aws/things/NAME/shadow/update"; char MQTT_PUB[] = "$aws/things/NAME/shadow/update"; Also set up the credentials of certificates of CA Certificates, Client Certificate and private Key downloaded earlier.
  4. Using the Pubsubclient library the M5Stack publishes data to IoT Cloud.

AWS SES

  1. Verify your email and set up the HTTPS API of AWS SES.
  2. The requested URL is of the form: https://EMAIL.us-east-1.amazonaws.comAction=SendEmail&Source=DESTINATIONEMAIL
  3. Sent the request with other parameters like the subject of mail and body of the mail.

Circuit Diagram

 

ESP32 Cam and Heart Rate Sensor Circuit Diagram

First the OV2640 Camera on ESP32Cam reads the image of the user's face and stores it in a buffer. That buffer is then transferred to neural network and processed. The Predictions are sent to M5COre2 via UART ( 13, 14 Pins on M5COre2). The Heart Rate sensor Data pin is connected to pin 36 of M5Stack Core2 and power is 5V and Ground The IR Distance sensor Data pin is connected to pin 35 of M5Stack Core2 and power is 5V and Ground.

Code
#include "esp_camera.h"
#include <WiFi.h>
#include "soc/soc.h"
#include "soc/rtc_cntl_reg.h"
#define CAMERA_MODEL_AI_THINKER
#include "camera_pins.h"
String inData;
void startCameraServer();
void setup() {
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0); //disable brownout detector
  Serial.begin(38400);
  Serial.setDebugOutput(true);
  Serial.println();
  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  //init with high specs to pre-allocate larger buffers
  if(psramFound()){
    config.frame_size = FRAMESIZE_240X240;
    config.jpeg_quality = 10;
    config.fb_count = 2;
  } else {
    config.frame_size = FRAMESIZE_240X240;
    config.jpeg_quality = 12;
    config.fb_count = 1;
  }
  // camera init
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    return;
  }
  //drop down frame size for higher initial frame rate
  sensor_t * s = esp_camera_sensor_get();
  s->set_framesize(s, FRAMESIZE_240X240);
  s->set_vflip(s, 0);  
}
void loop() {
  while(1){
     inData= Serial.readString();
    if(inData.length() != 0){
      startCameraServer();
    }
  }
}

M5Core2

#include <M5Core2.h>
#include <WiFiClientSecure.h>
#include <PubSubClient.h>
#include <ArduinoJson.h> //https://github.com/bblanchon/ArduinoJson
#include <time.h>
#define emptyString String()
#include "certificates.h"
const int MQTT_PORT = 8883;
const char MQTT_SUB_TOPIC[] = "$aws/things/" THINGNAME "/shadow/update";
const char MQTT_PUB_TOPIC[] = "$aws/things/" THINGNAME "/shadow/update";
uint8_t DST = 1;
WiFiClientSecure net;
WiFiClientSecure client_mail;
PubSubClient client(net);
#define samp_siz 4
#define rise_threshold 5
int sensorPin = 36;
double alpha=0.75;
int period=20;
double refresh=0.0;
#define RXp2 13
#define TXp2 14
String indata;
int counter=0;
void setup() {
  // put your setup code here, to run once:
  M5.begin();
  WiFi.hostname(THINGNAME);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);
  net.setTrustAnchors(&cert);
  net.setClientRSACert(&client_crt, &key);
  client.setServer(MQTT_HOST, MQTT_PORT);
  while (!client.connected())
  {
    if (client.connect(THINGNAME))
    {
      M5.Lcd.println("connected to mqtt");
    }
  }
  M5.Lcd.fillScreen(WHITE);
  M5.Lcd.setTextColor(BLACK , WHITE);
  M5.Lcd.setTextSize(2);
  Serial2.begin(38400, SERIAL_8N1, RXp2, TXp2);
  pinMode(35,INPUT); //IR
  pinMode(36, INPUT);  //Heart
}
void loop() {
  M5.Lcd.setTextColor(BLACK , GREEN);
  M5.Lcd.setTextSize(3);
  M5.Lcd.setCursor(20, 40);
  M5.Lcd.print("Welcome \n Mask and Heart Rate  \n Management System");
    if(digitalRead(35)==LOW){  
    M5.Lcd.clear(WHITE);
    M5.Lcd.setCursor(20, 40);
    M5.Lcd.print("Person Detected \n Reading Mask \n Stand in Front of Camera");
    counter=0;
    Serial2.println("1");
    while (Serial2.available() <= 0)
    {}
    while(1){
      indata=Serial2.readString();
      if(indata.length() == 0){
      break;  
      }
      counter+=indata.toInt();
    }
    M5.Lcd.clear(WHITE);
    if(counter>=4){
      M5.Lcd.setCursor(20, 40);
      M5.Lcd.setTextColor(GREEN , WHITE);
      M5.Lcd.print("Mask Detected Successfully");
      int heart_cnt=0,tmp=0;
      while(heart_cnt<50){
         static double oldValue=0;
         static double oldrefresh=0;
         int beat=analogRead(36);
         double value=alpha*oldValue+(0-alpha)*beat;
         refresh=value-oldValue;
         tmp=beat/13;
         M5.Lcd.setCursor(0, 80);
         M5.Lcd.printf("BP:%d\n",tmp);
         oldValue=value;
         oldrefresh=refresh;
         delay(period*13);
         heart_cnt++;
      }
      DynamicJsonDocument jsonBuffer(JSON_OBJECT_SIZE(3) + 100);
      JsonObject root = jsonBuffer.to<JsonObject>();
      JsonObject state = root.createNestedObject("state");
      JsonObject heart_reported = state.createNestedObject("heart");
      JsonObject mask_reported = state.createNestedObject("mask");
      heart_reported["value"] = tmp; //heart rate
      mask_reported["value"] = 1; //1 for correct mask
      Serial.printf("Sending  [%s]: ", MQTT_PUB_TOPIC);
      serializeJson(root, Serial);
      char shadow[measureJson(root) + 1];
      serializeJson(root, shadow, sizeof(shadow));
      if (!client.publish(MQTT_PUB_TOPIC, shadow, false))   //publish data
         M5.Lcd.println("Error Sending Data");
    }
    else{
      M5.Lcd.setCursor(20, 40);
      M5.Lcd.setTextColor(RED , WHITE);
      M5.Lcd.print("No Mask Detected");
      //sending ses email change capital words to your respective destination and authenticated email
      client_mail.connect("GET https://YOUREMAIL.us-west-2.amazonaws.com?Action=SendEmail&Source=YOUREMAIL&Destination.ToAddresses.member.1=DESTINATIONEMAIL&Message.Subject.Data=Mask%20Intruder%20Alert.&Message.Body.Text.Data=There%20Is%20Person%20at%20Gate%20without%20a%20Mask.");
    }
    delay(3000);
    M5.Lcd.clear(WHITE);
    }   
}
Video

Tags
Contest
india automation challenge 2021
ESP32
ESP32-CAM
amazon cloud
mqtt
M5Stack
Heart rate sensor
Covid19
Have any question realated to this Article?

Ask Our Community Members

WhatsApp
Telegram
Discord
Forum