Change your cover photo
Upload
pilaniashivankur91@gmail.com
Change your cover photo
This user account status is Approved

This user has not added any information to their profile yet.

OnTheGo: Voice enabled command and control
MAX78000 Feather Board

The project is aimed at providing an easy to use interface for people with locomotive disability and associated constraints that impedes their routine functions of moving about within the vicinity of their house for their tasks. The project will serve as an extension to the end user's ability to communicate with a central hub(The ReSpeaker core board and the ReSpeaker Mic Array) by virtue of speech processing to translate their commands to accomplish certain tasks on the go. The speech processing will make use of related peripherals on the MAX78000 board (Digital mic & Low power stereo audio codec) along with Natural Language Processing(NLP) to process audio from the end user. The audio will be captured and further processed to let you interact with your home appliances, your plant, your office, your internet-equipped devices or any other things in your daily life, The entire process from user audio capture to speech enabled device control is accomplished in a series of steps as follows:

1. Raw audio capture from end user by means of on board digital microphone on the MAX78000 wearable that will act as a slave device. Some preliminary filtering shall also be done onboard.

2. The second stage involves audio enhancement, keyword spotting from the captured audio, speech to command translation using NLP using certain audio processing algorithms such as Voice Activity Detection(VAD), Direction of Arrival(DOA) estimation, Beamforming, Acoustic Echo Cancellation(AEC) etc to generate clear and precise commands to accomplish a certain task. This process is done using the ReSpeaker Core Board and the ReSpeaker Far Field Mic Array module(already available). This device shall act as the master.

The wireless interface between the two will be implemented using a dedicated Wi-Fi or Bluetooth module (already available).

3. The final stage involves use of generated voice commands to control devices in the vicinity for routine or special tasks. While this project does not require the use of any camera as such, the onboard VGA camera or any extra hardware to this effect can enhance the use cases of this project.

The software component of this project will make use of a C application with certain components in assembly and javascript for interfacing.