Adaptive Musical Instrument for Physically Challenged Persons

This project aims to create an adaptive musical instrument that allows physically challenged individuals to express their musical creativity through touch-sensitive sensors, visual feedback via LED matrix displays, and sound output through speakers. Leveraging the Internet of Things (IoT) with Arduino Uno R4 WiFi, this instrument provides a highly interactive and accessible music-making experience. The low-cost, user-friendly design promotes inclusivity, enabling everyone to enjoy and pursue music, regardless of physical limitations.

Components Required

• Arduino Uno R4 WiFi

• MPR121 Capacitive Touch Sensor

• MAX7219 Dot Matrix Display (8x8 or larger)

• MP3-TF Module with Speaker

• DIY MIDI Interface with 6N138 Optocoupler

• Boost Converter (3.7V to 5V)

• Battery (3.7V LiPo, 4000 mAh)

• Resistors (220Ω, 10kΩ)

• Capacitors (0.1µF, 10µF, 100µF)

• Wires, Breadboard, Connectors - For prototyping and connections.

Circuit Diagram Explanation

• Arduino Uno R4 WiFi serves as the central controller, handling input from the MPR121 touch sensor and generating visual and audio feedback.

• MPR121 Touch Sensor is connected to the I2C pins of the Arduino (SDA/A4 and SCL/A5), allowing the user to trigger sounds by touching the conductive pads.

• MAX7219 Dot Matrix Display is connected via SPI communication (DIN to D11, CS to D10, CLK to D13) to provide dynamic visual feedback, enhancing the user’s interaction with the instrument.

• MP3-TF Module is connected to the Arduino through the SoftwareSerial interface on D2, playing pre-loaded sounds based on touch input. A 100µF capacitor is used in series with the speaker to block DC.

• DIY MIDI Interface with 6N138 Optocoupler connects to the MIDI IN of the Arduino, allowing it to send MIDI data to external devices. Resistors are used to protect the optocoupler and stabilize the output signal.

• Power Supply is managed by a boost converter, stepping up the 3.7V from a LiPo battery to 5V, ensuring stable power for all components.

Code Explanation

#include <Wire.h>
#include <Adafruit_MPR121.h>
#include <MD_MAX72XX.h>
#include <SoftwareSerial.h>
#include <MIDI.h>

Libraries Included:

• Wire.h: For I2C communication.

• Adafruit_MPR121.h: For interfacing with the MPR121 capacitive touch sensor.

• MD_MAX72XX.h: For controlling MAX7219 LED matrix displays.

• SoftwareSerial.h: For serial communication with the MP3-TF module.

• MIDI.h: For sending MIDI messages.

MPR121 Setup

Adafruit_MPR121 cap = Adafruit_MPR121();

MPR121 Object: Creates an instance of the MPR121 touch sensor class.

MAX7219 Setup

#define MAX_DEVICES 1
#define DATA_PIN 11
#define CS_PIN 10
#define CLK_PIN 13
MD_MAX72XX mx = MD_MAX72XX(CS_PIN, MAX_DEVICES);

MAX7219 Definitions:

• MAX_DEVICES: Number of MAX7219 ICs used.

• DATA_PIN, CS_PIN, CLK_PIN: Pins for data, chip select, and clock for the MAX7219.

• MD_MAX72XX mx: Creates an instance of the MAX7219 control class.

MP3-TF Module SetupSoftwareSerial mp3Serial(2, 3); // RX, TX for MP3 module

Software Serial: Sets up serial communication for the MP3-TF module using digital pins 2 (RX) and 3 (TX).

MIDI Setup

MIDI_CREATE_DEFAULT_INSTANCE();

MIDI Instance: Creates a default MIDI instance for MIDI communication.

Setup Function

void setup() {
Serial.begin(115200);
mp3Serial.begin(9600);

Serial Communication:

• Serial.begin(115200): Starts serial communication with a baud rate of 115200 for debugging.

• mp3Serial.begin(9600): Starts serial communication with the MP3-TF module at 9600 baud

if (!cap.begin(0x5A)) {
Serial.println("MPR121 not found, check wiring!");
while (1);
}

MPR121 Initialization:

• cap.begin(0x5A): Initializes the MPR121 with address 0x5A. If initialization fails, it prints an error message and halts execution

mx.begin();
mx.clear();

MAX7219 Initialization: 

• mx.begin(): Initializes the MAX7219.

• mx.clear(): Clears the display.

MIDI Initialization:

• MIDI.begin(MIDI_CHANNEL_OMNI): Initializes MIDI communication to listen to all channels.

mp3Serial.write(0x7E); // Send play command, as an example
}

Test MP3 Module:Sends a placeholder command to the MP3 module. This might need to be replaced with a specific command depending on your MP3 module’s protocol.

Loop Function

void loop() {
uint16_t touched = cap.touched();
for (uint8_t i = 0; i < 12; i++) {
if (touched & (1 << i)) {
mx.clear();
mx.setPoint(0, i, true); // Visual feedback
playSound(i); // Play sound corresponding to touch pad
sendMIDI(i); // Send MIDI note
}
}
}

Touch Detection:

• cap.touched(): Reads the touch input status.

• Loops through each of the 12 touch pads (0-11).

• if (touched & (1 << i)): Checks if the current pad is touched. 

mx.clear(): Clears the display.

mx.setPoint(0, i, true): Lights up the LED corresponding to the touched pad for visual feedback.

playSound(i): Calls a function to play a sound for the touched pad.

sendMIDI(i): Sends a MIDI note for the touched pad

void playSound(uint8_t index) {
mp3Serial.write(0x7E); // Replace with specific play commands
}

Play Sound Function: Sends a command to the MP3 module to play a specific sound. Replace 0x7E with the actual command for playing a track based on the index.

void sendMIDI(uint8_t note) {
MIDI.sendNoteOn(note + 60, 127, 1); // Note On, velocity 127, channel 1
delay(100);
MIDI.sendNoteOff(note + 60, 127, 1); // Note Off
}

Send MIDI Function:

MIDI.sendNoteOn(note + 60, 127, 1): Sends a MIDI note on message with the note number adjusted by 60 (to shift the note range), velocity 127, on channel 1.

• delay(100): Waits for 100 milliseconds.

• MIDI.sendNoteOff(note + 60, 127, 1): Sends a MIDI note off message for the same note.

This setup allows you to use touch pads to control visual feedback on an LED matrix, play sounds from an MP3 module, and send MIDI notes for musical output

CODE

#include <Wire.h>
#include <Adafruit_MPR121.h>
#include <MD_MAX72XX.h>
#include <SoftwareSerial.h>
#include <MIDI.h>

// MPR121 setup
Adafruit_MPR121 cap = Adafruit_MPR121();

// MAX7219 setup
#define MAX_DEVICES 1
#define DATA_PIN 11
#define CS_PIN 10
#define CLK_PIN 13
MD_MAX72XX mx = MD_MAX72XX(CS_PIN, MAX_DEVICES);

// MP3-TF module setup
SoftwareSerial mp3Serial(2, 3); // RX, TX for MP3 module

// MIDI setup
MIDI_CREATE_DEFAULT_INSTANCE();

void setup() {
Serial.begin(115200);
mp3Serial.begin(9600);

// Initialize MPR121
if (!cap.begin(0x5A)) {
Serial.println("MPR121 not found, check wiring!");
while (1);
}

// Initialize MAX7219
mx.begin();
mx.clear();

// Initialize MIDI
MIDI.begin(MIDI_CHANNEL_OMNI);

// Test MP3 module
mp3Serial.write(0x7E); // Send play command, as an example
}

void loop() {
// Check touch input
uint16_t touched = cap.touched();
for (uint8_t i = 0; i < 12; i++) {
if (touched & (1 << i)) {
mx.clear();
mx.setPoint(0, i, true); // Visual feedback
playSound(i); // Play sound corresponding to touch pad
sendMIDI(i); // Send MIDI note
}
}
}

// Function to play sound based on touch input
void playSound(uint8_t index) {
// Simple command to play track, depends on MP3-TF module command set
mp3Serial.write(0x7E); // Replace with specific play commands
}

// Function to send MIDI note based on touch input
void sendMIDI(uint8_t note) {
MIDI.sendNoteOn(note + 60, 127, 1); // Note On, velocity 127, channel 1
delay(100);
MIDI.sendNoteOff(note + 60, 127, 1); // Note Off
}

Click on the GitHub Image to view or download the code