ESP RainMaker with ESP32 – Voice Assistant Integration

Published  October 6, 2022   0
ESP RainMaker Voice Assistant Integration

In the last tutorial, we look at the ESP Rainmaker platform and how we can use it with the ESP32 through Arduino IDE. In this part of the tutorial, we will investigate the RainMaker cloud platform a bit deeper and will try to add voice assistant functionality to it. This setup can be used to build IoT based Smart home, Voice controlled home Appliances, home automation etc. Now let’s look at the ESP RainMaker API and its configurations. 

ESP RainMaker API Configuration

Device Types

A device is a logical user-controllable entity like a Switch, Lightbulb, Thermostat, Temperature Sensor, etc. A simple node will generally have a single device. However, an ESP32 can also have multiple devices like say 2 switches on a single board, or it can act as a bridge for other devices like BLE sensors. Each device must have a unique name within the node, and optionally, a type. All Standard predefined types supported by ESP RainMaker can be found by following the link.

In the last example, we used the Switch device to turn on and off the LED using either the hardware switch or the Rainmaker app. We can create any type of device with multiple function calls by adding the device definition and its different parameters. But the RainMaker API makes our job much easier by providing several predefined device types. These standard device types are also useful for defining some special handling in smartphone apps or other third-party integrations like Google Assistant or Alexa. The table below shows the standard device types, their parameters, Google Assistant and Alexa Handles, and their icons in the mobile app. Parameters in Bold are mandatory and * indicates the primary parameters.

ESP RainMaker API Device Type

Services

A service is an entity very similar to a device in terms of structure, with the main difference being that it is meant for operations that may not necessarily need to be user-visible. A typical example of this is OTA Firmware Upgrade Service, which can have parameters like URL, status, etc. You can find all the provided services and their parameters in the table below. The parameters in Bold are mandatory.

ESP RainMaker API Services

ESP RainMaker OTA service provides a simple interface to upgrade the firmware on the nodes remotely. At a high level, you just need to upload the firmware image on some secure web server (HTTPS) and provide the URL to the node. ESP RainMaker also offers a firmware image hosting service as well as a web dashboard to make the process simpler.

The Time service enables us to use either the network time or the Timezone and it is necessary to have features like Device Scheduling. Network time is usually fetched using SNTP. ESP RainMaker provides an abstraction layer over the SMTP component. It also provides a facility to set the Timezone or the POSIX Timezone.

The Internet of Things (IoT) primarily relates to devices connected to the Internet and naturally, exchanging data (like telemetry and commands) over the Internet. However, some automation, wherein the device itself acts independently is desired for many applications. Eg. turning on lights at 6 pm and turning them off at 11 pm. To support such automation, ESP RainMaker provides the scheduling service.

The System service enables us to perform system-related operations like reboot, WiFi reset or Factory reset from the phone apps.

Parameters

These are the control and monitoring parameters of a device. Eg. On/Off power state, brightness, current/target temperature, etc. All such parameters must have a unique name within the device. These parameters can also have additional fields like value, type, bounds, UI Information, etc. The parameter values can be boolean, integer, float, or string.

Just like the device types, the parameters in the API are also very important. The table below shows the different pre-defined parameters and their types in the RainMaker API. Knowing each type will make API usage much easier. These parameter types are related to any of the device types or the service.

ESP RainMaker API Parameters

UI Elements

Just like the pre-defined device types, the RainMaker API also offers some UI elements which will be shown in the phone apps. The following table shows these UI elements and their parameters. These elements define how the parameters should be rendered in the app.

ESP RainMaker API UI Elements

Google Voice Assistant and Alexa Integration with RainMaker

Nowadays voice assistant integrations like Alexa or Google Voice Assistant are a must for any IoT device. The ESP RainMaker provides both Alexa and Google Voice Assistant integration. This enables us to use our ESP32 with popular smart devices such as Amazon Eco or Google home line-ups or even from our smartphones with voice commands. So we can use this setup for Home automation

Now, let’s look at how we can integrate Google Voice Assistant and Alexa features into our program. For typical use cases like switches and lightbulbs, ESP RainMaker provides predefined types of devices and parameters. ESP RainMaker has now developed a layer that converts these parameters into formats that Alexa and GVA can understand. As a result, a device type in RainMaker (such as a light or switch) corresponds to a similar device type in the other system, and its attributes (such as power, brightness, color, saturation, intensity, etc.) are mapped to the appropriate functions or traits. We obtain a straightforward brightness-controllable light if all we have are the power and brightness parameters. Hue, saturation, and intensity when combined result in a color light in Google Voice Assistant and Alexa.

Circuit Diagram for the ESP RainMaker Example

As an example, we are going to read a sensor value from the ESP32 and also control an LED using RainMaker. We will also show you how to control the LED using Google Voice Assistant and Alexa. For that first, make the connection as per the circuit diagram given below. Connect an LED to the GPIO23 through a current limit resistor just like in the previous example. Then connect the DHT11 sensor to the GPIO21 and the VIN and GND.

ESP RainMaker Example Circuit Diagram

Here is how the real-life connection looks.

ESP RainMaker Example Circuit

Arduino Code for Using ESP RainMaker with Voice Assistant

The code we are going to use is very simple. As we discussed in the previous tutorial, we will create a node and we will add new devices with specific parameters to it. In this example, we will create 3 devices namely my_switch, temperature, and humidity. The my_switch device will be used to control the LED and the other two will be used to monitor and display temperature and humidity from the DHT11 sensor. First, make sure to install the necessary libraries. The necessary libraries needed for this example are SimpleTimer library by Alexander Kiryanenko and the DHT sensor library by Adafruit. You can install them directly from the library manager by searching their names.

Once it’s done, copy the Arduino code provided at the bottom of this article and paste it into a new Arduino Sketch. Compile it and upload it to the ESP32. After a successful upload, do the provision using the RainMaker App. You can follow our previous tutorial if you are facing any doubts about how to set up and upload the code.

Alexa Integration

To use Alexa voice assistant, we must configure it with the RainMaker account. To do that open the RainMaker app and go to the Settings Page. On the settings page select Voice Services. On the Voice Services page select Amazon Alexa and click on the link with the Amazon Alexa button.

RainMaker App Voice Assistant Integration

Now the app will redirect you to the Amazon login page. Log in with your Amazon account, which is used with other Alexa devices. Once logged in, you will be redirected to a second login page use your RainMaker credentials here and log in. That’s it. your RainMaker account is now linked to your Amazon account. Now you can use your Alexa devices or app to control the ESP32.

RainMaker App Voice Assistant Integration Google and Alexa

Google Voice Assistant Integration

To use the RainMaker firmware with Google Assistant, we must install the Google Home App. Once installed and logged into the Google Home app, click on the plus icon on the app’s home screen and select Setup a device. Then select the works with google option on the next screen.

RainMaker App Google Voice Assistant Integration

Now you will be presented with the list of all Google assistant supported devices. In the list select the ESP RainMaker option. You can also use the search function to find it easily. Now you will be redirected to the ESP RainMaker login page. Once logged in to your device, a shortcut will be added to the home screen and you’re ready to use it with the Google Assistant feature. You use either the shortcut or your voice to control the LED.

ESP RainMaker App Google Voice Assistant Integration

Code Explanation

Now let’s investigate the code itself. At the start, we have included all the necessary libraries, which include the WiFi and RainMaker, Simple timer, and DHT libraries. Then we defined a few variables to store the LED status, temperature, and humidity. Later we specified the BLE credentials using the pointers service_name and pop, which will be used while the provisioning. Then we have defined the GPIO pins used for the LED, switch, and DHT sensor. We have also created the DHT and simple timer instances to use i=with the code. Also created 3 devices named Temperature, Humidity, and LED to use with the RainMaker.

#include "RMaker.h"
#include "WiFi.h"
#include "WiFiProv.h"
#include "DHT.h"
#include <SimpleTimer.h>
#include <wifi_provisioning/manager.h>
// Set Defalt Values
#define DEFAULT_LED_MODE false
#define DEFAULT_Temperature 0
#define DEFAULT_Humidity 0
// BLE Credentils
const char *service_name = "RainMaker_BLE";
const char *pop = "1234567";
// GPIO
static uint8_t gpio_reset = 0;
static uint8_t DHTPIN = 21;
static uint8_t led = 23;
bool led_state = false;
bool wifi_connected = 0;
DHT dht(DHTPIN, DHT11);
SimpleTimer Timer;
//------------------------------------------- Declaring Devices -----------------------------------------------------//
//The framework provides some standard device types like switch, lightbulb, fan, temperature sensor.
static TemperatureSensor temperature("Temperature");
static TemperatureSensor humidity("Humidity");
static Switch my_switch("LED", &led);
                    The sysProvEvent function is used for system provisioning events. This function will obtain the Wi-Fi credentials and connect ESP32 to the router.
void sysProvEvent(arduino_event_t *sys_event) {
  switch (sys_event->event_id) {
    case ARDUINO_EVENT_PROV_START:
#if CONFIG_IDF_TARGET_ESP32
      Serial.printf("\nProvisioning Started with name \"%s\" and PoP \"%s\" on BLE\n", service_name, pop);
      printQR(service_name, pop, "ble");
#else
      Serial.printf("\nProvisioning Started with name \"%s\" and PoP \"%s\" on SoftAP\n", service_name, pop);
      printQR(service_name, pop, "softap");
#endif
      break;
    case ARDUINO_EVENT_WIFI_STA_CONNECTED:
      Serial.printf("\nConnected to Wi-Fi!\n");
      wifi_connected = 1;
      delay(500);
      break;
    case ARDUINO_EVENT_PROV_CRED_RECV: {
        Serial.println("\nReceived Wi-Fi credentials");
        Serial.print("\tSSID : ");
        Serial.println((const char *) sys_event->event_info.prov_cred_recv.ssid);
        Serial.print("\tPassword : ");
        Serial.println((char const *) sys_event->event_info.prov_cred_recv.password);
        break;
      }
  }
}

The write_callback function will be responsible for checking the particular device which sent the data and then changing the parameter of the device accordingly. This callback function will be used for the device LED. If the device is ‘LED’ and its parameter is ‘Power’ then this function will check whether the state of the LED pin is HIGH or LOW. If the power is HIGH, the LED will turn ON. Similarly, if the power is LOW, the LED will turn OFF.

void write_callback(Device *device, Param *param, const param_val_t val, void *priv_data, write_ctx_t *ctx) {
  const char *device_name = device->getDeviceName();
  Serial.println(device_name);
  const char *param_name = param->getParamName();
  if (strcmp(device_name, "LED") == 0)  {
    if (strcmp(param_name, "Power") == 0)    {
      Serial.printf("Received value = %s for %s - %s\n", val.val.b ? "true" : "false", device_name, param_name);
      led_state = val.val.b;
      (led_state == false) ? digitalWrite(led, LOW) : digitalWrite(led, HIGH);
      param->updateAndReport(val);
    }
  }
}

Inside the setup() function we initialized the serial communication at a baud rate of 115200. Then we configure the reset pin as an input pin and the LED pin as an output pin. This is done by using the pinMode() function and specifying the GPIO pin as the first parameter and the mode as the second parameter. Moreover, we will initially set the LED pin to a default state, which we have defined at the start. After that, the DHT sensor library is also initialized using the dht.begin function. Then we will declare a node with the name “ESP RainMaker Node”. Later we declared the ESP RainMaker node. Then attached all three devices to the node. We also declared the callback function for the my_switch device and enabled the OTA, Timezone, and Schedule services. Then we created a timer to send sensor data to the ESP RainMaker Cloud every two seconds. Later we started the RainMaker band using Rmaker.start function and called the sysProEvent function after that for provisioning.

void setup() {
  Serial.begin(115200);
  // Configure the input GPIOs
  pinMode(gpio_reset, INPUT);
  pinMode(led, OUTPUT);
  digitalWrite(led, DEFAULT_LED_MODE);
//Initialize DHT sensor
  dht.begin();
//------------------------------------------- Declaring Node -----------------------------------------------------//
  Node my_node;
  my_node = RMaker.initNode("ESP RainMaker");
//Standard switch device
  my_switch.addCb(write_callback);
 //------------------------------------------- Adding Devices in Node -----------------------------------------------------//
  my_node.addDevice(temperature);
  my_node.addDevice(humidity);
  my_node.addDevice(my_switch);
 //This is optional
  RMaker.enableOTA(OTA_USING_PARAMS);
  //If you want to enable scheduling, set time zone for your region using setTimeZone().
  //The list of available values are provided here https://rainmaker.espressif.com/docs/time-service.html
  // RMaker.setTimeZone("Asia/Shanghai");
  // Alternatively, enable the Timezone service and let the phone apps set the appropriate timezone
  RMaker.enableTZService();
  RMaker.enableSchedule();
  Serial.printf("\nStarting ESP-RainMaker\n");
  RMaker.start();
  // Timer for Sending Sensor's Data
  Timer.setInterval(2000);
  WiFi.onEvent(sysProvEvent);
#if CONFIG_IDF_TARGET_ESP32
  WiFiProv.beginProvision(WIFI_PROV_SCHEME_BLE, WIFI_PROV_SCHEME_HANDLER_FREE_BTDM, WIFI_PROV_SECURITY_1, pop, service_name);
#else
  WiFiProv.beginProvision(WIFI_PROV_SCHEME_SOFTAP, WIFI_PROV_SCHEME_HANDLER_NONE, WIFI_PROV_SECURITY_1, pop, service_name);
#endif
}

In the loop, we will monitor the state of GPIO0 and will update the parameter led_state accordingly. A long press of three seconds will reset the WiFi credentials and a long press of ten seconds will factory reset the ESP32 RainMaker firmware. These functions will be used fully while reconfiguring the device. The timer function is used to call the Send_senor function to update the sensor values to the cloud. The timer will call the function every two seconds, and once the function is executed the timer is reset to zero. This will repeat and will upload the sensor data periodically.  We will also print the status change to the serial monitor for debugging purposes.

void loop() {
  if (Timer.isReady() && wifi_connected) {                    // Check is ready a second timer
    Serial.println("Sending Sensor's Data");
    Send_Sensor();
    Timer.reset();                        // Reset a second timer
  }
//-----------------------------------------------------------  Logic to read button
  // Read GPIO0 (external button to reset device
  if (digitalRead(gpio_reset) == LOW) { //Push button pressed
    Serial.printf("Reset Button Pressed!\n");
    // Key debounce handling
    delay(100);
    int startTime = millis();
    while (digitalRead(gpio_reset) == LOW) delay(50);
    int endTime = millis();
    if ((endTime - startTime) > 10000) {
      // If key pressed for more than 10secs, reset all
      Serial.printf("Reset to factory.\n");
      wifi_connected = 0;
      RMakerFactoryReset(2);
    } else if ((endTime - startTime) > 3000) {
      Serial.printf("Reset Wi-Fi.\n");
      wifi_connected = 0;
      // If key pressed for more than 3secs, but less than 10, reset Wi-Fi
      RMakerWiFiReset(2);
    } else {
          // Toggle device state
          led_state = !led_state;
          Serial.printf("Toggle State to %s.\n", led_state ? "true" : "false");
          my_switch.updateAndReportParam(ESP_RMAKER_DEF_POWER_NAME, led_state);
          (led_state == false) ? digitalWrite(led, LOW) : digitalWrite(led, HIGH);
      }
  }
  delay(100);
}

The Send _Sensor function will read the sensor values from the DHT sensor and update it to the cloud upon calling.

void Send_Sensor() {
  float h = dht.readHumidity();
  float t = dht.readTemperature();
  Serial.print("Temperature - ");
Serial.println(t);
Serial.print("Humidity - ");
Serial.println(h);
  temperature.updateAndReportParam("Temperature", t);
  humidity.updateAndReportParam("Temperature", h);
}

And here is the result.

Supporting Files

Code
#include "RMaker.h"
#include "WiFi.h"
#include "WiFiProv.h"
#include "DHT.h"
#include <SimpleTimer.h>
#include <wifi_provisioning/manager.h>

// Set Defalt Values
#define DEFAULT_LED_MODE true
#define DEFAULT_Temperature 0
#define DEFAULT_Humidity 0

// BLE Credentils
const char *service_name = "RainMakerBLE";
const char *pop = "1234567";

// GPIO
static uint8_t gpio_reset = 0;
static uint8_t DHTPIN = 21;
static uint8_t led = 23;
bool led_state = true;

bool wifi_connected = 0;

DHT dht(DHTPIN, DHT11);

SimpleTimer Timer;

//------------------------------------------- Declaring Devices -----------------------------------------------------//

//The framework provides some standard device types like switch, lightbulb, fan, temperature sensor.
static TemperatureSensor temperature("Temperature");
static TemperatureSensor humidity("Humidity");
static Switch my_switch("LED", &led);

void sysProvEvent(arduino_event_t *sys_event)
{
  switch (sys_event->event_id) {
    case ARDUINO_EVENT_PROV_START:
#if CONFIG_IDF_TARGET_ESP32
      Serial.printf("\nProvisioning Started with name \"%s\" and PoP \"%s\" on BLE\n", service_name, pop);
      printQR(service_name, pop, "ble");
#else
      Serial.printf("\nProvisioning Started with name \"%s\" and PoP \"%s\" on SoftAP\n", service_name, pop);
      printQR(service_name, pop, "softap");
#endif
      break;
    case ARDUINO_EVENT_WIFI_STA_CONNECTED:
      Serial.printf("\nConnected to Wi-Fi!\n");
      wifi_connected = 1;
      delay(500);
      break;
    case ARDUINO_EVENT_PROV_CRED_RECV: {
        Serial.println("\nReceived Wi-Fi credentials");
        Serial.print("\tSSID : ");
        Serial.println((const char *) sys_event->event_info.prov_cred_recv.ssid);
        Serial.print("\tPassword : ");
        Serial.println((char const *) sys_event->event_info.prov_cred_recv.password);
        break;
      }
  }
}

void write_callback(Device *device, Param *param, const param_val_t val, void *priv_data, write_ctx_t *ctx)
{
  const char *device_name = device->getDeviceName();
  Serial.println(device_name);
  const char *param_name = param->getParamName();

  if (strcmp(device_name, "LED") == 0)
  {
    if (strcmp(param_name, "Power") == 0)
    {
      Serial.printf("Received value = %s for %s - %s\n", val.val.b ? "true" : "false", device_name, param_name);
      led_state = val.val.b;
      (led_state == false) ? digitalWrite(led, LOW) : digitalWrite(led, HIGH);
      param->updateAndReport(val);
    }
  }
}

void setup()
{

  Serial.begin(115200);

  // Configure the input GPIOs
  pinMode(gpio_reset, INPUT);
  pinMode(led, OUTPUT);
  digitalWrite(led, DEFAULT_LED_MODE);

  //Beginning Sensor
  dht.begin();

  //------------------------------------------- Declaring Node -----------------------------------------------------//
  Node my_node;
  my_node = RMaker.initNode("ESP RainMaker");

  //Standard switch device
  my_switch.addCb(write_callback);

  //------------------------------------------- Adding Devices in Node -----------------------------------------------------//
  my_node.addDevice(temperature);
  my_node.addDevice(humidity);
  my_node.addDevice(my_switch);

  //This is optional
  RMaker.enableOTA(OTA_USING_PARAMS);
  //If you want to enable scheduling, set time zone for your region using setTimeZone().
  //The list of available values are provided here https://rainmaker.espressif.com/docs/time-service.html
  // RMaker.setTimeZone("Asia/Shanghai");
  // Alternatively, enable the Timezone service and let the phone apps set the appropriate timezone
  RMaker.enableTZService();
  RMaker.enableSchedule();

  Serial.printf("\nStarting ESP-RainMaker\n");
  RMaker.start();

  // Timer for Sending Sensor's Data
  Timer.setInterval(3000);

  WiFi.onEvent(sysProvEvent);

#if CONFIG_IDF_TARGET_ESP32
  WiFiProv.beginProvision(WIFI_PROV_SCHEME_BLE, WIFI_PROV_SCHEME_HANDLER_FREE_BTDM, WIFI_PROV_SECURITY_1, pop, service_name);
#else
  WiFiProv.beginProvision(WIFI_PROV_SCHEME_SOFTAP, WIFI_PROV_SCHEME_HANDLER_NONE, WIFI_PROV_SECURITY_1, pop, service_name);
#endif

}

void loop()
{

  if (Timer.isReady() && wifi_connected) {                    // Check is ready a second timer
    Serial.println("Sending Sensor's Data");
    Send_Sensor();
    Timer.reset();                        // Reset a second timer
  }


  //-----------------------------------------------------------  Logic to read button

  // Read GPIO0 (external button to reset device
  if (digitalRead(gpio_reset) == LOW) { //Push button pressed
    Serial.printf("Reset Button Pressed!\n");
    // Key debounce handling
    delay(100);
    int startTime = millis();
    while (digitalRead(gpio_reset) == LOW) delay(50);
    int endTime = millis();

    if ((endTime - startTime) > 10000) {
      // If key pressed for more than 10secs, reset all
      Serial.printf("Reset to factory.\n");
      wifi_connected = 0;
      RMakerFactoryReset(2);
    } else if ((endTime - startTime) > 3000) {
      Serial.printf("Reset Wi-Fi.\n");
      wifi_connected = 0;
      // If key pressed for more than 3secs, but less than 10, reset Wi-Fi
      RMakerWiFiReset(2);
    } else {
          // Toggle device state
          led_state = !led_state;
          Serial.printf("Toggle State to %s.\n", led_state ? "true" : "false");
          my_switch.updateAndReportParam(ESP_RMAKER_DEF_POWER_NAME, led_state);
          (led_state == false) ? digitalWrite(led, LOW) : digitalWrite(led, HIGH);
      }
  }
  delay(100);
}

void Send_Sensor()
{
  float h = dht.readHumidity();
  float t = dht.readTemperature();

  Serial.print("Temperature - ");
  Serial.println(t);
  Serial.print("Humidity - ");
  Serial.println(h);

  temperature.updateAndReportParam("Temperature", t);
  humidity.updateAndReportParam("Temperature", h);
}
Have any question realated to this Article?

Ask Our Community Members