Real Time Face Detection and Tracking Robot using Arduino

Published  June 13, 2017   45
Aswinth Raj
Author
Real Time Face Detection and Tracking Project using Arduino

Ever wanted to build a Face Tracking Robotic Arm or Robot by simply using Arduino and not any other programming like OpenCV, visual basics C# etc? Then read along, in this project we are going to implement face detection by blending in the power of Arduino and Android. In this project, the mobile camera will move along with your face with the help of servos. The advantage of using the Android Mobile Phone here is that you do not need to invest on a camera module and the whole image detection work can be done in the phone itself, you do not need your Arduino connected to your computer for this to work. Here we have used Bluetooth Module with Arduino to communicate with Mobile wirelessly.

The Android application used in this project was created using Processing Android, you can either directly install the application by downloading the APK file (read further for link) or put on your programming cap and make your own more appealing Android Application using the Processing Code given further in the Tutorial. Learn more about Processing by checking our previous Processing Projects.

By the end of this tutorial you will have a Mini Tilt and Span Robotic Arm that could track your face and move along with it. You can use this (with further advancement) to record your vlog videos or even take a selfie with the rear camera of your mobile phone since it positions your face exactly at the centre of your mobile screen. So!! Sounds interesting? Check the Demo Video at the end this tutorial to see it working. Let’s see how we can build one...

I have tried my best to make this project to work as simple as possible, anyone with minimum knowledge on hardware or coding can use this guidelines to make this project work in no time. However once you make it I suggest you to get behind the codes so that you can really know what makes this thing work and how. 

Face-Tracking-Robotic-With-Arduino-and-Android-bluetooth

 

Materials Required:

  1. Arduino Nano
  2. Servo motor SG90 – 2Nos
  3. Android Phone with decent camera
  4. HC-05/HC-06 Bluetooth Module
  5. Computer for programming
  6. 3D printer (optional)
  7. 9V Battery

 

3D Printing the Required Parts (Optional):

In order to pan and tilt our mobile phone we need some mechanical structures like a mobile holder and a few servo brackets. You can use a cardboard to make one, since I have a 3D printer I decided to 3D print these parts.

3D printing is an amazing tool that can contribute a lot when building prototype projects or experimenting with new mechanical designs. If you have not yet discovered the benefits of a 3D printer or how it works you can read The beginners Guide to 3D printing.

If you own or have access to a 3D printer then you can use the STL files which can be downloaded from here to directly print and assemble them. However few parts like the mobile phone holder might need some modifications based on the dimensions of your phone. I have designed it for my MOTO G mobile phone. I have used a very basic printer of mine to print all the parts. The printer is FABX v1 from 3ding which comes at an affordable price with a print volume of 10 cubic cm. The cheap price comes with a trade off with low print resolution and no SD card or print resuming function. I am using software called Cura to print the STL files. The settings that I used to print the materials are given below you can use the same or change them based on your printer.

3d-printer settings for printing parts for Arduino face tracking robot

 

Once you print all the required materials you can secure them in position by using screws and some hot glue. After you assembly is complete it should look something like this below.

3d printed parts for Arduino face tracking robot 3d printed parts for Arduino face tracking robot

3d printed parts for Arduino face tracking robot 3d-printed parts for Arduino face tracking robot

 

Schematic and Hardware:

The Circuit for this Face Tracking on Smart Phone project is shown in the image below:

Face Tracking Robot With Arduino and Android circuit diagram

The Circuit Consists of two Servo motors, out of which one is used to move the mobile phone left/right and the other is used to tilt the mobile phone up/down. The direction in which the servo has to move will be instructed by the Arduino Nano which itself gets information from the Bluetooth (HC-05) module. The whole circuit is powered by a 9V battery.

This circuit can be connected easily on your breadboard or you can also solder these on a small Perf board like I have done here.

 

Setting up your Android Application:

As, I said earlier the main brain working behind this project is this Android application. This android application was developed using Processing Android. You can directly install this application on your mobile phone and launch that by following the steps below.

  1. Download the APK file from here.
  2. Power on the circuit shown above.
  3. In your phone settings search for Bluetooth module named “HC-05”
  4. If you have named it something else other than “HC-05” change it back to HC-05 since only then the application will work.
  5. Pair with your Bluetooth module with the password “1234” or “0000”.
  6. Now, launch the Application in portrait mode. You should see your camera screen and also “Connected to: HC-05” on the top of your screen.
  7. Try moving your camera over a face and a green box should appear on top of it and its position will also be displayed on the top left corner of your screen as shown below.

tracking pictures using arduino and android phone with bluetooth

You can take this Arduino Face Tracking Project to next level by bringing in lot of advancements for which you won’t need to code your own Android application. Creating an Android application might sound difficult but trust me with the help of Processing you can learn it in no time. The complete processing code that is used to build this application can be downloaded here. You are free to make any advancement with your own creativity. Check below projects to learn more about Processing:

 

Programming your Arduino:

The Android application will detect the face and its position on screen; it will then decide which direction it should move based on the position of the face so that the face gets to the centre of the screen. This direction is then sent to the Arduino via Bluetooth Module.

The Arduino program for this project is fairly simple, we just have to control the direction of the two servo motors based on the values received from the Bluetooth Module. The complete code can be found at the end of this tutorial, I have also explained few important lines below.

 

Below line of code establishes a serial connection with pins D12 as RX and D11 as TX. Hence the pin D12 must be connected to the TX of the BT module and the pin D11 to the RX of the BT module.

SoftwareSerial cam_BT(12, 11);  // RX, TX

 

Then we have initialised the Bluetooth module at a baud rate of 9600. Make sure you module also works on the same baud rate. Else change it accordingly.

cam_BT.begin(9600); //start the Bluetooth communication at 9600 baudrate
cam_BT.println("Ready to take commands");

 

Below line reads what is coming in through the Bluetooth module. Also the data is saved in the variable “BluetoothData”.

if (cam_BT.available()) //Read whats coming in through Bluetooth
   {
BluetoothData=cam_BT.read();
Serial.print("Incoming from BT:");
Serial.println(BluetoothData);
}

 

Based on the data received from the Bluetooth the motors direction is controlled. To turn a motor left the motor is decrement by a value of 2 from its previous position. You can increase this value 2 to 4 or 6 if you need the arm to move faster. But, it might create some jerks making the camera unstable.

if (BluetoothData==49) //Turn Left
{pos1+=2; servo1.write(pos1);}

if (BluetoothData==50) //Turn Right
{pos1-=2; servo1.write(pos1);}

if (BluetoothData==51) //Turn Up
{pos2-=2; servo2.write(pos2);}

if (BluetoothData==52) //Turn Down
{pos2+=2; servo2.write(pos2);}

 

Working:

Once we are ready with our hardware, code and Android Application its time for some action. Simply power your Arduino and open the android application. The Application will automatically connect to the HC-05 (must be named HC-05) Bluetooth module and will wait for a face to be detected. Simply place the phone in our mobile holder and sit in front of it. You should notice your servo motors moving your phone so that your face will be placed at the centre of the screen.  Now move around within the range of the camera and your mobile phone will follow your movements. You can also try it by placing and moving any picture.

Face Tracking Robotic With Arduino and processing Android

The complete working of the project is shown in the video below. You can build a lot more on top of it which is left for your creativity, I hope you enjoyed the project and got it working. If not, leave your feedbacks in the comment section and I will respond to them.

Check out the arduino projects section on the website to find more such projects.

Code

/*Arduino Code for Face Tracking Arduino
 * Coded by Circuitdigest.com
 * On 25-05-2017
 */

 /*CONNECTION DETIALS
  * Arduino D11 -> RX of BT Module
  * Arduino D12 -> Tx of BT
  * Servo1 -> pin 3 of arduino Nano to pan
  * Servo2 -> pin 5 of arduino Nano to tilt
  */

#include <Servo.h> //header to drive servo motors
#include <SoftwareSerial.h>// import the serial library
SoftwareSerial cam_BT(12, 11);  // RX, TX
int ledpin=13; // led on D13 will show blink on / off
int BluetoothData; // the data given from Computer

//lets declare the servo objects
Servo servo1;  
Servo servo2;

long gmotor,gnum1,gnum2;
int pos;
int flag=0; 
int pos1 = 40;
int pos2 = 90;

void setup() {
   servo1.attach(3);
   servo2.attach(5);;

   //**Initial position of all four servo motors**//
   servo1.write(pos1);
   servo2.write(pos2);
   //**initialised**//
   
cam_BT.begin(9600); //start the Bluetooth communication at 9600 baudrate
cam_BT.println("Ready to take commands");

Serial.begin(57600); 
Serial.println("Face tracking programming by CircuitDigest.com");
}

//***Function for each Servo actions**//
void call(int motor, int num1, int num2) // The values like Motor number , from angle and to angle are received 
{
Serial.println("Passing values...");
flag =0;
switch (motor) 
  {  
    case 1:            // For motor one
    { 
    Serial.println("Executing motor one");
    if(num1<num2) // Clock wise rotation
    { 
        for ( pos =num1; pos<=num2; pos+=1)
            {
            servo1.write(pos);
            delay( 20);
            }
    }
    
    if(num1>num2) // Anti-Clock wise rotation
    {
        for ( pos =num1; pos>=num2; pos-=1)
        {
        servo1.write(pos);
        delay( 20);
        }
    }
     break;
    }     

       ////////JUST  DUPLICATE FOR OTHER SERVOS////

       
    case 2:    // For motor 2
    {
    Serial.println("Executing motor two");
    if(num1<num2)
    {
        for ( pos =num1; pos<=num2; pos+=1)
        {
        servo2.write(pos);
        delay( 20);
        }
    }
    if(num1>num2)
    {
        for ( pos =num1; pos>=num2; pos-=1)
        {
        servo2.write(pos);
        delay( 20);
        }
    }
    break;
    }     
  }
}

void loop() {
 if(Serial.available()>0) //Read whats coming in through Serial
  {
  gmotor= Serial.parseInt(); 
  Serial.print(" selected Number-> ");  
  Serial.print(gmotor);
  Serial.print(" , ");    
  gnum1= Serial.parseInt();   
  Serial.print(gnum1);  
  Serial.print(" degree , ");
  gnum2= Serial.parseInt();   
  Serial.print(gnum2);  
  Serial.println(" degree ");
  flag=1;
  }
   
 if (cam_BT.available()) //Read whats coming in through Bluetooth
 {
  BluetoothData=cam_BT.read();
  Serial.print("Incoming from BT:");
  Serial.println(BluetoothData);
 }

 if (flag ==1)
 call(gmotor,gnum1,gnum2); //call the respective motor for action

 if (BluetoothData==49) //Turn Left
 {pos1+=2; servo1.write(pos1);}

 if (BluetoothData==50) //Turn Right
 {pos1-=2; servo1.write(pos1);}

 if (BluetoothData==51) //Turn Up
 {pos2-=2; servo2.write(pos2);}

 if (BluetoothData==52) //Turn Down
 {pos2+=2; servo2.write(pos2);}

 flag=0;
 BluetoothData=0;
}

Video

Have any question realated to this Article?

Ask Our Community Members

Comments

Submitted by Irving Cosillo on Wed, 07/19/2017 - 22:24

Permalink

Hello!

Your app is very cool!! I am trying to replicate your code, but I cannot make the Arduino part to work. I did the following, maybe you can tell me what I am doing wrong:

I have downloaded the APK file and installed it in my Android Phone.
I have downloaded and uploaded the Arduino code in my Arduino.
I have connected the BT module in the right way.
I have paired the BT device with my phone.
The app shows that it is connected to my BT device and displays his address.
The led of the BT device change his state and appears to be paired.

The problem is that it is not receiving any data from the app. On the terminal it is not showing anything.

Sorry, for the late reply. But if you are still facing the issue. I can help you out.

You should first try to sort out where the problem is. Use a normal android application like bluetooth Terminal to link with arduino and check if you are receiving anyting.

 

Submitted by Omar on Sat, 08/26/2017 - 07:35

Permalink

Hello! I do everything you said , i downnload the apk file and i have connected the BT module in the right way , but there is no connection between the app and the bleutooth ( i rename it hc-05) , can you help me and tell me what the problem ?

Submitted by Omar on Mon, 09/04/2017 - 10:13

Permalink

Hello Raj , thank you , i change the name to hc-06 and it works only one time , then it stop , can you give me other solution , and i want to ask you if the version of android or the phone can make a problem with this application , thank youa

You can use any android phone running android version kitkat or above.

The application should work properly after renaming to "HC-06". There is no other solution to this other. What do you mean by "works only one time"?. Is you application crashing?

What error message are you getting? 

Submitted by Omar on Mon, 09/04/2017 - 19:47

Permalink

I am not receiving any message , but i think the application has a problem since it works one time , then i close the app and open it second time it does not work ( there is no connection with the bleutooth module) can you send me the apk file here to download it, maybe it work this time

Submitted by SHIVANI U PATHAK on Fri, 09/15/2017 - 08:07

Permalink

Hello
I wanna know if this code is full I wanna replicate this project.plz let me know. Thank you.

Submitted by Naseef on Sat, 09/16/2017 - 00:29

Permalink

Hi. I just wan to ask, what if there are multiple faces? How does it react? I mean, can it detect two or more faces simultaneously?

Hi Naaseef,

Yes it can detect two or mare faces. But, in that case the robotic arm will try to swing between both the faces and end up in covering either one. I never considered this situation during programming. However, the complete code is given and hence you can tweak it as you desire 

Thank you for replying. My application on phone doesn't work I compiled it n it has no error but on phone it say this app is not responding I am having a blank screen for few seconds and thn it stops responding. I also wanted to know if I can still launch this app without the hardwares just to see if it works, or its gonna show the error.?

The app on start up tries to establish a Bluetooth connection with any device named "HC-06".  If the connection is not established it will show a blank screen for some time and then crash. 

However, there is a hack turn on Bluetooth on your friends mobile and name it as HC-06, now pair with his/her deivce from you phone. After pairing launch the application and it should work 

Submitted by pooja on Tue, 10/10/2017 - 04:42

Permalink

I tried doing it n launched the app somehow it still shows it crashed and its irresponsive .. do u think its my android device? or some other problem can I contact u ?

I am pretty much active on the forums as well on the comment section you can reach me here. The application should no longer be crashing. Either your version of android is not supporting or there might be some error in the program as I dint not test it on all devices.

However try the following, Connect your phone to your computer, launch processiong android mode on your computer and then launch the Android application. When your android application crashes there should some error message on the processing console this messsage should help you to slve the issue

 

Submitted by Hari on Wed, 10/11/2017 - 17:35

Permalink

Hi. Nice work there bro. I wanna know if there's any way to connect the Android via A USB, because I currently don't have a Bluetooth module. If it's not possible via USB, can you suggest the best Bluetooth module? Thanks for the help...

Replacing the Bluetooth module with a USB cable is not impossible but a bit less efficient and more complex. I would personally not recommend that method. You can use any normal bluetooth module for this project the "HC-05" or "HC-06" will be a best choice

Submitted by Mohamed on Sun, 12/10/2017 - 18:43

Permalink

i need a similar system and the difference between the another system and this ... OR the final Report For this one ... Thanks

Submitted by glen on Thu, 01/04/2018 - 09:51

Permalink

i get this error when i run on device:
FATAL EXCEPTION: main
Process: processing.test.face_tracking_android_code, PID: 25691
java.lang.RuntimeException: Unable to start activity ComponentInfo{processing.test.face_tracking_android_code/processing.test.face_tracking_android_code.MainActivity}: java.lang.IllegalStateException: You need to use a Theme.AppCompat theme (or descendant) with this activity.
at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2452)
.........
what am i missing?

Submitted by Elijah on Sat, 01/06/2018 - 11:35

Permalink

Good day sir. I tried to replicate this project. I just want to know why does the app doesn't detect any face? It says connected to hc - 05 but it doesnt detect faces. Please respond to my message thank you

Submitted by Mango on Sat, 01/06/2018 - 18:09

Permalink

Hi. Your project is awesome! We have tried to duplicate it, But we're having a trouble. The servo motors don't move at all, even though we've done all the instructions. We could also see on our phone that we are connected to the HC-05 Bluetooth module. And the green box for the face recognition but the servos don't do anything. Please reply back. Really need some help. Thank youuu so much!

Submitted by yusuf on Sat, 02/17/2018 - 13:34

Permalink

Hi, i have try to tracked a face with this project, because i don't have sucessful.
My servo motor don't rotate. But i have download the programme for arduino and the software for my phone, and my circuit is correct but this project don't work.
HELP ME PLEASE!

Submitted by AKhilesh on Sun, 04/01/2018 - 11:33

Permalink

How can I get a app bro.It is not getting in the apk file
How can I get it

Submitted by Shubham aggarwal on Mon, 04/09/2018 - 09:03

Permalink

The black wire of servo motors that are connected to each other should also be connected to ground(gnd) on arduino board...!!!

Submitted by Mukhritdin on Tue, 06/19/2018 - 13:16

Permalink

thank you for an awesome project! but I have set everything like you explained but still, I cannot move the motors. So what problem would it be? thanks in advance

Submitted by Angel on Wed, 06/20/2018 - 00:58

Permalink

:20: error: 'gmotor' does not name a type

gmotor largo, gnum1, gnum2;

^

43: error: 'llamada' does not name a type

llamada vacia (int motor, int num1, int num2) // Se reciben los valores como numero de motor, desde angulo y angel

Submitted by muhamed on Sat, 07/07/2018 - 17:19

Permalink

can i replace a phone by a simple camera ??

Hello! I would like to use your project in order to track the number of people that are using another project of mine. Do you have any idea how could I keep in a base the people that are recognised?