Real Time Face Detection and Tracking Project using Arduino

Ever wanted to build a Face Tracking Robotic Arm or Robot by simply using Arduino and not any other programming like OpenCV, visual basics C# etc? Then read along, in this project we are going to implement face detection by blending in the power of Arduino and Android. In this project, the mobile camera will move along with your face with the help of servos. The advantage of using the Android Mobile Phone here is that you do not need to invest on a camera module and the whole image detection work can be done in the phone itself, you do not need your Arduino connected to your computer for this to work. Here we have used Bluetooth Module with Arduino to communicate with Mobile wirelessly.

The Android application used in this project was created using Processing Android, you can either directly install the application by downloading the APK file (read further for link) or put on your programming cap and make your own more appealing Android Application using the Processing Code given further in the Tutorial. Learn more about Processing by checking our previous Processing Projects.

By the end of this tutorial you will have a Mini Tilt and Span Robotic Arm that could track your face and move along with it. You can use this (with further advancement) to record your vlog videos or even take a selfie with the rear camera of your mobile phone since it positions your face exactly at the centre of your mobile screen. So!! Sounds interesting? Check the Demo Video at the end this tutorial to see it working. Let’s see how we can build one...

I have tried my best to make this project to work as simple as possible, anyone with minimum knowledge on hardware or coding can use this guidelines to make this project work in no time. However once you make it I suggest you to get behind the codes so that you can really know what makes this thing work and how. 

Face-Tracking-Robotic-With-Arduino-and-Android-bluetooth

 

Materials Required:

  1. Arduino Nano
  2. Servo motor SG90 – 2Nos
  3. Android Phone with decent camera
  4. HC-05/HC-06 Bluetooth Module
  5. Computer for programming
  6. 3D printer (optional)
  7. 9V Battery

 

3D Printing the Required Parts (Optional):

In order to pan and tilt our mobile phone we need some mechanical structures like a mobile holder and a few servo brackets. You can use a cardboard to make one, since I have a 3D printer I decided to 3D print these parts.

3D printing is an amazing tool that can contribute a lot when building prototype projects or experimenting with new mechanical designs. If you have not yet discovered the benefits of a 3D printer or how it works you can read The beginners Guide to 3D printing.

If you own or have access to a 3D printer then you can use the STL files which can be downloaded from here to directly print and assemble them. However few parts like the mobile phone holder might need some modifications based on the dimensions of your phone. I have designed it for my MOTO G mobile phone. I have used a very basic printer of mine to print all the parts. The printer is FABX v1 from 3ding which comes at an affordable price with a print volume of 10 cubic cm. The cheap price comes with a trade off with low print resolution and no SD card or print resuming function. I am using software called Cura to print the STL files. The settings that I used to print the materials are given below you can use the same or change them based on your printer.

3d-printer settings for printing parts for Arduino face tracking robot

 

Once you print all the required materials you can secure them in position by using screws and some hot glue. After you assembly is complete it should look something like this below.

3d printed parts for Arduino face tracking robot 3d printed parts for Arduino face tracking robot

3d printed parts for Arduino face tracking robot 3d-printed parts for Arduino face tracking robot

 

Schematic and Hardware:

The Circuit for this Face Tracking on Smart Phone project is shown in the image below:

Face Tracking Robot With Arduino and Android circuit diagram

The Circuit Consists of two Servo motors, out of which one is used to move the mobile phone left/right and the other is used to tilt the mobile phone up/down. The direction in which the servo has to move will be instructed by the Arduino Nano which itself gets information from the Bluetooth (HC-05) module. The whole circuit is powered by a 9V battery.

This circuit can be connected easily on your breadboard or you can also solder these on a small Perf board like I have done here.

 

Setting up your Android Application:

As, I said earlier the main brain working behind this project is this Android application. This android application was developed using Processing Android. You can directly install this application on your mobile phone and launch that by following the steps below.

  1. Download the APK file from here.
  2. Power on the circuit shown above.
  3. In your phone settings search for Bluetooth module named “HC-05”
  4. If you have named it something else other than “HC-05” change it back to HC-05 since only then the application will work.
  5. Pair with your Bluetooth module with the password “1234” or “0000”.
  6. Now, launch the Application in portrait mode. You should see your camera screen and also “Connected to: HC-05” on the top of your screen.
  7. Try moving your camera over a face and a green box should appear on top of it and its position will also be displayed on the top left corner of your screen as shown below.

tracking pictures using arduino and android phone with bluetooth

You can take this Arduino Face Tracking Project to next level by bringing in lot of advancements for which you won’t need to code your own Android application. Creating an Android application might sound difficult but trust me with the help of Processing you can learn it in no time. The complete processing code that is used to build this application can be downloaded here. You are free to make any advancement with your own creativity. Check below projects to learn more about Processing:

 

Programming your Arduino:

The Android application will detect the face and its position on screen; it will then decide which direction it should move based on the position of the face so that the face gets to the centre of the screen. This direction is then sent to the Arduino via Bluetooth Module.

The Arduino program for this project is fairly simple, we just have to control the direction of the two servo motors based on the values received from the Bluetooth Module. The complete code can be found at the end of this tutorial, I have also explained few important lines below.

 

Below line of code establishes a serial connection with pins D12 as RX and D11 as TX. Hence the pin D12 must be connected to the TX of the BT module and the pin D11 to the RX of the BT module.

SoftwareSerial cam_BT(12, 11);  // RX, TX

 

Then we have initialised the Bluetooth module at a baud rate of 9600. Make sure you module also works on the same baud rate. Else change it accordingly.

cam_BT.begin(9600); //start the Bluetooth communication at 9600 baudrate
cam_BT.println("Ready to take commands");

 

Below line reads what is coming in through the Bluetooth module. Also the data is saved in the variable “BluetoothData”.

if (cam_BT.available()) //Read whats coming in through Bluetooth
   {
BluetoothData=cam_BT.read();
Serial.print("Incoming from BT:");
Serial.println(BluetoothData);
}

 

Based on the data received from the Bluetooth the motors direction is controlled. To turn a motor left the motor is decrement by a value of 2 from its previous position. You can increase this value 2 to 4 or 6 if you need the arm to move faster. But, it might create some jerks making the camera unstable.

if (BluetoothData==49) //Turn Left
{pos1+=2; servo1.write(pos1);}

if (BluetoothData==50) //Turn Right
{pos1-=2; servo1.write(pos1);}

if (BluetoothData==51) //Turn Up
{pos2-=2; servo2.write(pos2);}

if (BluetoothData==52) //Turn Down
{pos2+=2; servo2.write(pos2);}

 

Working:

Once we are ready with our hardware, code and Android Application its time for some action. Simply power your Arduino and open the android application. The Application will automatically connect to the HC-05 (must be named HC-05) Bluetooth module and will wait for a face to be detected. Simply place the phone in our mobile holder and sit in front of it. You should notice your servo motors moving your phone so that your face will be placed at the centre of the screen.  Now move around within the range of the camera and your mobile phone will follow your movements. You can also try it by placing and moving any picture.

Face Tracking Robotic With Arduino and processing Android

The complete working of the project is shown in the video below. You can build a lot more on top of it which is left for your creativity, I hope you enjoyed the project and got it working. If not, leave your feedbacks in the comment section and I will respond to them.

Check out the arduino projects section on the website to find more such projects.

Code: 

/*Arduino Code for Face Tracking Arduino
 * Coded by Circuitdigest.com
 * On 25-05-2017
 */

 /*CONNECTION DETIALS
  * Arduino D11 -> RX of BT Module
  * Arduino D12 -> Tx of BT
  * Servo1 -> pin 3 of arduino Nano to pan
  * Servo2 -> pin 5 of arduino Nano to tilt
  */

#include <Servo.h> //header to drive servo motors
#include <SoftwareSerial.h>// import the serial library
SoftwareSerial cam_BT(12, 11);  // RX, TX
int ledpin=13; // led on D13 will show blink on / off
int BluetoothData; // the data given from Computer

//lets declare the servo objects
Servo servo1;  
Servo servo2;

long gmotor,gnum1,gnum2;
int pos;
int flag=0; 
int pos1 = 40;
int pos2 = 90;

void setup() {
   servo1.attach(3);
   servo2.attach(5);;

   //**Initial position of all four servo motors**//
   servo1.write(pos1);
   servo2.write(pos2);
   //**initialised**//
   
cam_BT.begin(9600); //start the Bluetooth communication at 9600 baudrate
cam_BT.println("Ready to take commands");

Serial.begin(57600); 
Serial.println("Face tracking programming by CircuitDigest.com");
}

//***Function for each Servo actions**//
void call(int motor, int num1, int num2) // The values like Motor number , from angle and to angle are received 
{
Serial.println("Passing values...");
flag =0;
switch (motor) 
  {  
    case 1:            // For motor one
    { 
    Serial.println("Executing motor one");
    if(num1<num2) // Clock wise rotation
    { 
        for ( pos =num1; pos<=num2; pos+=1)
            {
            servo1.write(pos);
            delay( 20);
            }
    }
    
    if(num1>num2) // Anti-Clock wise rotation
    {
        for ( pos =num1; pos>=num2; pos-=1)
        {
        servo1.write(pos);
        delay( 20);
        }
    }
     break;
    }     

       ////////JUST  DUPLICATE FOR OTHER SERVOS////

       
    case 2:    // For motor 2
    {
    Serial.println("Executing motor two");
    if(num1<num2)
    {
        for ( pos =num1; pos<=num2; pos+=1)
        {
        servo2.write(pos);
        delay( 20);
        }
    }
    if(num1>num2)
    {
        for ( pos =num1; pos>=num2; pos-=1)
        {
        servo2.write(pos);
        delay( 20);
        }
    }
    break;
    }     
  }
}

void loop() {
 if(Serial.available()>0) //Read whats coming in through Serial
  {
  gmotor= Serial.parseInt(); 
  Serial.print(" selected Number-> ");  
  Serial.print(gmotor);
  Serial.print(" , ");    
  gnum1= Serial.parseInt();   
  Serial.print(gnum1);  
  Serial.print(" degree , ");
  gnum2= Serial.parseInt();   
  Serial.print(gnum2);  
  Serial.println(" degree ");
  flag=1;
  }
   
 if (cam_BT.available()) //Read whats coming in through Bluetooth
 {
  BluetoothData=cam_BT.read();
  Serial.print("Incoming from BT:");
  Serial.println(BluetoothData);
 }

 if (flag ==1)
 call(gmotor,gnum1,gnum2); //call the respective motor for action

 if (BluetoothData==49) //Turn Left
 {pos1+=2; servo1.write(pos1);}

 if (BluetoothData==50) //Turn Right
 {pos1-=2; servo1.write(pos1);}

 if (BluetoothData==51) //Turn Up
 {pos2-=2; servo2.write(pos2);}

 if (BluetoothData==52) //Turn Down
 {pos2+=2; servo2.write(pos2);}

 flag=0;
 BluetoothData=0;
}

Video: 

Comments (29)

  • B.Aswinth Raj's picture
    B.Aswinth Raj

    Hi Vicente,

    You project was awesome. You must have spent a lot of time in getting that app running!! I am not a good App developer so I used the processing environment to just create a bare minimum application for the project. Also mine does not recognize faces it can only detect faces. 

    Thanks for sharing!!

    Jun 21, 2017
  • qucai's picture
    qucai

    hi. how you get it the apps. i can`t understand

    Jun 21, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    The app was created by using Processing Android. You can either use my code by downloading it here 

    Or you can simply download the APK file here and install it on your mobile phone.

    Read the article for complete details

    Jun 22, 2017
  • Irving Cosillo's picture
    Irving Cosillo

    Hello!

    Your app is very cool!! I am trying to replicate your code, but I cannot make the Arduino part to work. I did the following, maybe you can tell me what I am doing wrong:

    I have downloaded the APK file and installed it in my Android Phone.
    I have downloaded and uploaded the Arduino code in my Arduino.
    I have connected the BT module in the right way.
    I have paired the BT device with my phone.
    The app shows that it is connected to my BT device and displays his address.
    The led of the BT device change his state and appears to be paired.

    The problem is that it is not receiving any data from the app. On the terminal it is not showing anything.

    Jul 19, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    Sorry, for the late reply. But if you are still facing the issue. I can help you out.

    You should first try to sort out where the problem is. Use a normal android application like bluetooth Terminal to link with arduino and check if you are receiving anyting.

     

    Aug 14, 2017
  • Omar's picture
    Omar

    Hello! I do everything you said , i downnload the apk file and i have connected the BT module in the right way , but there is no connection between the app and the bleutooth ( i rename it hc-05) , can you help me and tell me what the problem ?

    Aug 26, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    Hi Omar,

    Sorry for the late reply. But, can you try naming your Bluetooth Device as "HC-06" and let me know if it works.

    Thanks.

    Sep 01, 2017
  • Omar's picture
    Omar

    Hello Raj , thank you , i change the name to hc-06 and it works only one time , then it stop , can you give me other solution , and i want to ask you if the version of android or the phone can make a problem with this application , thank youa

    Sep 04, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    You can use any android phone running android version kitkat or above.

    The application should work properly after renaming to "HC-06". There is no other solution to this other. What do you mean by "works only one time"?. Is you application crashing?

    What error message are you getting? 

    Sep 04, 2017
  • Omar's picture
    Omar

    I am not receiving any message , but i think the application has a problem since it works one time , then i close the app and open it second time it does not work ( there is no connection with the bleutooth module) can you send me the apk file here to download it, maybe it work this time

    Sep 04, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    This problem is common with HC-05 bluetooth modules. If this problem occurs just close all your apps. Unpair your Bluetooth Module and Pair it again. The APK file attached in the article is working and tested.

    Thanks

    Sep 06, 2017
  • Omar's picture
    Omar

    Please send me the apk file here to download it, maybe it will work

    Sep 04, 2017
  • SHIVANI U PATHAK's picture
    SHIVANI U PATHAK

    Hello
    I wanna know if this code is full I wanna replicate this project.plz let me know. Thank you.

    Sep 15, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    Yes the complete code is given here. you can easily replicate it

    Sep 15, 2017
  • Naseef's picture
    Naseef

    Hi. I just wan to ask, what if there are multiple faces? How does it react? I mean, can it detect two or more faces simultaneously?

    Sep 16, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    Hi Naaseef,

    Yes it can detect two or mare faces. But, in that case the robotic arm will try to swing between both the faces and end up in covering either one. I never considered this situation during programming. However, the complete code is given and hence you can tweak it as you desire 

    Sep 17, 2017
  • pooja's picture
    pooja

    Its awesome concept but the app is not working do you know how i can make it work?

    Oct 04, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    I can help only if I know what problem you are facing with the app.  Install processing, conect the phone to lap, open processing and launch the program. If your application has crached the crash report will appea on the processing console 

    Oct 04, 2017
  • pooja's picture
    pooja

    Thank you for replying. My application on phone doesn't work I compiled it n it has no error but on phone it say this app is not responding I am having a blank screen for few seconds and thn it stops responding. I also wanted to know if I can still launch this app without the hardwares just to see if it works, or its gonna show the error.?

    Oct 05, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    The app on start up tries to establish a Bluetooth connection with any device named "HC-06".  If the connection is not established it will show a blank screen for some time and then crash. 

    However, there is a hack turn on Bluetooth on your friends mobile and name it as HC-06, now pair with his/her deivce from you phone. After pairing launch the application and it should work 

    Oct 05, 2017
  • pooja's picture
    pooja

    I tried doing it n launched the app somehow it still shows it crashed and its irresponsive .. do u think its my android device? or some other problem can I contact u ?

    Oct 10, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    I am pretty much active on the forums as well on the comment section you can reach me here. The application should no longer be crashing. Either your version of android is not supporting or there might be some error in the program as I dint not test it on all devices.

    However try the following, Connect your phone to your computer, launch processiong android mode on your computer and then launch the Android application. When your android application crashes there should some error message on the processing console this messsage should help you to slve the issue

     

    Oct 10, 2017
  • Hari's picture
    Hari

    Hi. Nice work there bro. I wanna know if there's any way to connect the Android via A USB, because I currently don't have a Bluetooth module. If it's not possible via USB, can you suggest the best Bluetooth module? Thanks for the help...

    Oct 11, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    Replacing the Bluetooth module with a USB cable is not impossible but a bit less efficient and more complex. I would personally not recommend that method. You can use any normal bluetooth module for this project the "HC-05" or "HC-06" will be a best choice

    Oct 12, 2017
  • pooja's picture
    pooja

    Hello, can you tell me what version of android are you using that could support the application ?

    Oct 17, 2017
  • pooja's picture
    pooja

    I am getting only one error that says the value of the parameter who is not used. can you help me solve it please ..Thank you

    Oct 18, 2017
  • B.Aswinth Raj's picture
    B.Aswinth Raj

    I cannot understand the error. Can you copy paste it here?

    Oct 18, 2017

Leave a comment