Virtual Reality using Arduino and Processing

Arduino

ByAswinth Raj 112

Virtual Reality Project using Arduino and Processing

This is a very interesting project in which we are going to learn how to implement virtual reality using Arduino and Processing. For most of us, the movie Iron man by Jon Favreau has always been an inspiration to build new things that will make our life easy and more fun. I have personally admired the Techs that are shown in the movie and have always wanted to build something similar to that. So, in this project I have tried to mimic the Virtual reality stuffs that happen in the movie, like we can simply wave our hand in front of the computer and move the pointer to the desired location and perform some tasks.

Here I will show you how you can simply wave your hand in front of webcam and draw something on your computer. I will also show you how you can toggle lights by virtually moving your hand and making clicks with your fingers in the air.

 

Concept:

To make this happen we have to leverage the power of Arduino and Processing combined. Most of would be familiar with Arduino, but Processing might be new for you. Processing is an application just like Arduino and it is also Open source and free to download. Using Processing you can create simple system applications, Android applications and much more. It also has the ability to do Image Processing and Voice recognition. It is just like Arduino and is much easy to learn, but do not worry if you are completely new to processing because  I have written this tutorial fairly simple so that anyone with interest can make this working in no time.

In this tutorial we are using Processing to create a simple System application which provides us an UI and track the position of our hand using Image processing. Now, we have to make left click and right click using our fingers. To make that happen I have used two hall sensors (one on my index finger and the other on middle finger) which will be read by the Arduino Nano. The Arduino also transmits the click status to the Computer wirelessly via Bluetooth.

It might sound complicated but, Trust me; it is not as hard as it sounds. So let us take a look at the materials needed for this project to be up and running.

 

Materials Required:

  1. Arduino Nano
  2. Hall sensor (A3144) – 2Nos
  3. A small piece of magnet
  4. Bluetooth Module (HC-05/HC-06)
  5. 9V battery
  6. Connecting Wires Dot board.
  7. A pair of gloves
  8. Arduino IDE (Software)
  9. Processing IDE(Software)
  10. A Computer with Webcam and Bluetooth (you can also use external Bluetooth or Webcam for your computer)

 

Schematics and Hardware:

The hardware part of this project is very simple and easy to build. The complete schematic is shown below.

Circuit Diagram for Virtual Reality Project using Arduino

The Arduino, resistors and the berg stick pins are soldered onto a dot board as shown below.

arduino nano

The hall sensor and the Bluetooth module is soldered to a connector wire as shown below. 

hall sensor and bluetooth module

Once these two sections are ready it can be assembled onto gloves so that it is easy to use. I have used disposable plastic gloves which can be purchased from any medical shop near you. You should make sure that the magnet comes on your thumb finger and the hall sensor 1 and hall sensor 2 should be present before your index and middle finger respectively. I have used duck tapes to secure the components in place. Once the components are assembled it should look something like this.

Virtual Reality Gloves with Bluetooth and Hall Sensor
Now let us open the Arduino IDE and start programming.

 

Program for Arduino:

The purpose of this Arduino code is it to read the status of the hall sensor and broadcast them using the Bluetooth module. It should also receive data from Bluetooth and toggle the onboard LED based on the incoming value. The complete program is given at the end of this tutorial; I have also explained few lines below.

if (Phs1!=HallState_1 || Phs2!=HallState_2) //Check if new keys are pressed
 {
if (HallState_1==LOW && HallState_2==LOW)
Aisha.write(1);
if (HallState_1==HIGH && HallState_2==LOW)
Aisha.write(2);
if (HallState_1==LOW && HallState_2==HIGH)
Aisha.write(3);
if (HallState_1==HIGH && HallState_2==HIGH)
Aisha.write(4);
 }

As shown in the above lines based on the status of the hall sensor the Bluetooth will write a particular value. For example if hall sensor 1 is high and hall sensor 2 is low, then we will broadcast the vale “2” via the Bluetooth module. Make sure you write the values to the BT module and not print them. Because it will be easy to read the only on Processing side only if they are written. Also the value will only send if it is not as same as the previous value.

if (BluetoothData=='y')
digitalWrite(ledpin,HIGH);
if (BluetoothData=='n')
digitalWrite(ledpin,LOW);

These lines are used to toggle the onboard LED which is connected to the Pin 13, based on the value receive by the BT module. For example if the module receives a ‘y’ then the LED is turned on and if it receives an ‘n’ then it is turned off.

 

Program for processing:

The purpose of the Processing program is to create a system application which can act as an UI (User interface) and also perform image processing to track a particular object. In this case we track the blue object that we stuck to our gloves above. The program basically has four screens.

  1. Calibration Screen
  2. Main Screen
  3. Paint Screen
  4. LED toggle Screen

We can navigate from one screen to another by simply waving our hands and dragging screens on air. We can also make clicks on desired places to toggle LED or even draw something on screen.

You can copy paste the complete Processing program (given at the end) and modify it based on your creativity or simple download the EXE files from here, and follow the following steps to launch the application.

  1. Install JAVA in your computer if you have not installed it before
  2. Install You Cam perfect on your computer
  3. Power up your Arduino and pair your Computer with the Bluetooth Module
  4. Launch the application file

If everything goes fine you should be able to notice the LED on your Bluetooth module getting stable and your webcam light going ON. If you have any problems reach me through the comment section and I will help you out.

Watch the video at the end to know how to calibrate your application and use it.

If you want to modify the code and build more features into this then you can use the following insights of the program

The processing IDE can be downloaded from here. If you want to learn more about processing and create more interesting projects then you can visit the tutorials here.  

Processing has the ability to read Serial data, in this program the serial data is comes from the Bluetooth COM port. You have to select which COM port your Bluetooth is connect to by using this line below

port = new Serial(this,Serial.list()[1],9600);

Here I have selected my 1st COM port which is COM5 in my case (see image below) and I have mentioned that by Bluetooth module runs on 9600 baudrate.

VR

As said earlier processing also has the ability to do image processing, in this tutorial the images are sent inside the sketch using a webcam. In each image we track for a particular object. To know more about this you can visit this tutorial.

I have tried my best to explain the program (given at the end) through the comment lines. You can download the files here.

If you want to know more about the sketch you can reach me through the comment section and I will help you out.

Working:

Once the Hardware and software is ready, wear the gloves and get ready for some action. Now, simply power the Arduino and then launch the Application. The led on the Bluetooth module should go stable. Now it means that your System application has established a Bluetooth link with your Arduino.

You will get the following screen where you have to select the object to be tracked. This tracing can be simply done by clicking on the object. In this case the object is the Blue disc. Now you can move your object and notice that the pointer follows your object. Use a unique colour object and a bright room for best results.

Virtual Reality using processing

Now touch your thumb finger with index finger and you should see the message “Key 1 Pressed” and the when you press your thumb with middle finger you should see “Key 2 Pressed” this indicates that everything works fine and the calibration is over. Now click on the Done button.

Virtual Reality using arduino nano

Virtual Reality using processing image

Once the Done button is pressed you will be directed to the main screen where you can paint on air or toggle the LED on the Arduino Board as shown in the Video below. 

Code

#include <SoftwareSerial.h>// import the serial library

SoftwareSerial Aisha(11, 12); // TX, RX
int ledpin=13; // led on D13 will show blink on / off
int hall_1=9;
int hall_2=10;
int BluetoothData; // the data given from Computer
int HallState_1,HallState_2;
int change;
int Phs1,Phs2;
 
 void setup()
 {
   Aisha.begin(9600); //Bluetooth Module works at 9600 baudrate
   pinMode(ledpin,OUTPUT); //led pin as output
   pinMode(hall_1,INPUT); //hall sensor 1 as input
   pinMode(hall_2,INPUT); //hall snesor 2 is also input
 }
 
 void loop()
 {
   if (Aisha.available()) //if data is sent from laptop
   BluetoothData=Aisha.read(); //read it and store it in BluetoothData

 Phs1=HallState_1;
 Phs2=HallState_2;
 HallState_1 = digitalRead(hall_1);
 HallState_2 = digitalRead(hall_2);
 
 
 if (Phs1!=HallState_1 || Phs2!=HallState_2) //Check if new keys are pressed
 {
if (HallState_1==LOW && HallState_2==LOW)
Aisha.write(1);

if (HallState_1==HIGH && HallState_2==LOW)
Aisha.write(2);

if (HallState_1==LOW && HallState_2==HIGH)
Aisha.write(3);

if (HallState_1==HIGH && HallState_2==HIGH)
Aisha.write(4);
 }

if (BluetoothData=='y')
digitalWrite(ledpin,HIGH);
if (BluetoothData=='n')
digitalWrite(ledpin,LOW);

 
  }

//----------- Arduino code ends---------------//

 

 

 

 

//------------Processing code starts-----------//

import processing.video.*;  // Import Librarey to use video 
import processing.serial.*; //Import Librarey to use Serial Port (Bluetooth)

//**Global Variable Declarations**//
Serial port; //port is an object variable for Serial communication
int data; 
boolean calibration= false;
int mirror =0;
int mirrorn =-1;
PImage Done,Aisha,Paint,LED_Toggle,LED_on,LED_off;
boolean key1,key2,key3,movePaint,PaintScreen,PaintScreenClear,moveLED,LEDscreen;
float Paintx,Painty,avgX,avgY,LEDx,LEDy;
int count;
PImage img = createImage(380, 290, RGB);
int Px,Py;
Capture video;  //create an object named video
color trackColor;  //variable to store the color that we are going to track
float threshold = 50  ; //can be varied by the user
//_____End of variable declaration______//

//*Function to load all the images from data folder of the sketch*//
void loadImages()
{
  Done = loadImage("Done.png"); 
  Aisha = loadImage ("Aisha.png");
  Paint = loadImage("Paint.png");
  LED_Toggle = loadImage("LED_Toggle.png");
  LED_on = loadImage("LED_on.png");
  LED_off = loadImage ("LED_off.png");
}
//_____End of variable declaration______//

//**Executes only ones**//
void setup() {
  size(800, 600);
  loadImages();
  String[] cameras = Capture.list();
  printArray(cameras);
  video = new Capture(this, cameras[34]);
  video.start();
  key1=key2=key3=false;
  Paintx=width/10;
  Painty=height/8.5;
  LEDx=width/1.1;
  LEDy=height/8.5;
  movePaint=PaintScreen=PaintScreenClear=moveLED=LEDscreen=false;
  port = new Serial(this,Serial.list()[1],9600);
  println(Serial.list()); 
}
//**End of Setup**//

//**Triggered to update each frame of the video**//
void captureEvent(Capture video)  //when a new image comes in
 { video.read(); } //reas it as a video

//*Function to point which color to Track*//
void Calibrate()
{
   image(video,0,0);
   imageMode(CORNERS);
   image(Done,width/1.2,height/1.1,width,height); //position of the Done button
   if (mouseX>width/1.2 && mouseY>height/1.1) //If mouse is within the Done button
   {
   calibration=true;
   cursor(HAND);
   mirrorn=1;
   mirror=width;
   }

fill(#1B96E0);
textSize(20);

if (key1==true) //if hall sensor 1 is active on Arduino
text("Key-1 Pressed",width/12,height/1.05); //Text and its position
if (key2==true) //if hall sensor 2 is active on Arduino
text("Key-2 Pressed",width/12,height/1.05); //Text and its position
}
//_____End of Calibration______//

//*Function to represent the main Screen*//
void UI()
{
    imageMode(CORNERS);
    image(Aisha,0,0,width,height);
    imageMode(CENTER);
    
    if ((avgX<(width/10+((width/4)/2)) && avgY<(height/8.5+((height/4)/2)) && key1==true) || (movePaint==true&&key1==true)) //if clicked inside the image
    {
    movePaint=true;
    image (Paint, avgX,avgY,width/4, height/4); //Drag the image
    }
    else if (movePaint==false)
    image (Paint, Paintx,Painty,width/4, height/4); //place the image at corner
    else
      PaintScreen=true; 
      
       if ((avgX>(width/1.1-((width/4)/2)) && avgY<(height/8.5+((height/4)/2)) && key1==true) || (moveLED==true&&key1==true)) //if clicked inside the image
    {
    moveLED=true;
    image (LED_Toggle, avgX,avgY,width/4, height/4); //Drag the image
    }
    else if (moveLED==false)
    image (LED_Toggle, LEDx,LEDy,width/4, height/4); //place the image at corner
    else
      LEDscreen=true; 
}
//_____End of main screen function______//

//*Function to represent the Paint Screen*//
void Paintfun()
{
    imageMode(CENTER);
    background(#0B196A);
    image (Paint, width/2,height/2,width/1.5, height); 
   
img.loadPixels();
for (int IX = 210, Px=0; IX<=590; IX++, Px++)
{
for (int IY = 85, Py=0; IY<=375; IY++, Py++)
  {
  if ((dist(avgX,avgY,IX,IY)<4)  && key1==true)
  img.pixels[(Px+(Py*img.width))] = color(255);  //color of the paint background updated
  if (key2==true)
  PaintScreen = false;
  }
}
img.updatePixels();

image(img, width/2, height/2.6);
}
//_____End of main Paintscreen function______//

//*Function to display Toggle LED screen*//
void LEDfun()
{
  imageMode(CENTER);
  background(255);
  image(LED_on,(width/2 - width/4), height/3,width/4, height/5);
  image(LED_off,(width/2 + width/4), height/3,width/4, height/5);
   textSize(50);
  textAlign(CENTER);
  if (key1==true && avgX<300 && avgY>150 && avgX>95 && avgY<260)
  { fill(#751EE8);
  text("LED turned on",width/2,height/1.5);
  port.write(121);
  }
  if (key1==true && avgX<700 && avgY>150 && avgX>500 && avgY<260)
  { fill(#FC0808);
  text("LED turned   off",width/2,height/1.5);
  port.write(110);
  }
}
//_____End of main LEDscreen function_____//

//*Function to know which key is pressed*//
void key_select() {
  
  switch(data){
  case 1: 
     key1=true; key2=true;
     break;
     
  case 2: 
     key1=false; key2=true;
     break;
     
  case 3: 
     key1=true; key2=false;
     break;
     
  case 4: 
     key1=false; key2=false;
     break;
}
}
//_____End of function______//

 

void draw() {
  if (port.available()>0) //if there is an incoming BT value
  {
    data=port.read(); //read the BT incoming value and save in data
    println(key1,key2,data); //print for debugging
    key_select(); //toggle the variable key 1 and key2
  }
  
  video.loadPixels();
  if (calibration==false) //no calibration done
  Calibrate(); //Calibrate Screen
  if (calibration==true && (PaintScreen==false || LEDscreen==false) )
  UI(); //Main Screen
  if (PaintScreen==true && calibration ==true)
  Paintfun(); //Paint Screen
  if (LEDscreen==true && calibration ==true)
  LEDfun(); //LED toffle screen
  
 if (key2==true)
  movePaint=PaintScreen=PaintScreenClear=moveLED=LEDscreen=false; //go back to main screen
 

avgX = avgY = count = 0;

  // Begin loop to walk through every pixel
  for (int x = 0; x < video.width; x++ ) {
    for (int y = 0; y < video.height; y++ ) {
      int loc = x + y * video.width;
      // What is current color
      color currentColor = video.pixels[loc];
      float r1 = red(currentColor);
      float g1 = green(currentColor);
      float b1 = blue(currentColor);
      float r2 = red(trackColor);
      float g2 = green(trackColor);
      float b2 = blue(trackColor);

      float d = distSq(r1, g1, b1, r2, g2, b2); 

      if (d < threshold*threshold) {
        stroke(255);
        strokeWeight(1);
       // point((mirror-x)*mirrorn, y);
        avgX += x;
        avgY += y;
        count++;
      }
    }
  }

  if (count > 0) { 
    avgX = avgX / count;
    avgY = avgY / count;
    // Draw a circle at the tracked pixel
    fill(#21FADB);
    avgX = (mirror-avgX)*mirrorn;
    ellipse(avgX, avgY, 15, 15);

  } 
}
float distSq(float x1, float y1, float z1, float x2, float y2, float z2) {
  float d = (x2-x1)*(x2-x1) + (y2-y1)*(y2-y1) +(z2-z1)*(z2-z1);
  return d;
}

 

void mousePressed() {
  if(calibration==false)
  {
  int loc = mouseX + mouseY*video.width;
  trackColor = video.pixels[loc]; //load the color to be tracked
   }
}

Video

Comments

  • Shubham's picture
    Shubham
    May 18, 2017

    Hello sir i saw your your project demo can you just add something new codes and program it for using it on windows

  • Shivam's picture
    Shivam
    May 18, 2017

    Sir what is here
    done.png
    Aisha.png
    paint.png
    LED_ON.png etc

  • Aswinth Raj's picture
    Aswinth Raj
    May 19, 2017

    Hi shivam,

    I suppose that you copied the processing code and tried compiling it on your computer. In that case it might have shown you that those files were missing. 

    done.png
    Aisha.png
    paint.png
    LED_ON.png etc

    All the above are the names of the image files which I have used in the processing program. These images can be found in the data folder of the Processing sketch. To get these images download the code from here

    https://circuitdigest.com/sites/default/files/inlineimages/VR%20code%20f...

    Go to VR code files -> Virtual_Reality_POC -> data...

    Hope this answered your question!!!

     

     

  • Satyam 's picture
    Satyam
    Feb 01, 2018

    Where to put these images because it is still showing missing.plzz help

  • sulav's picture
    sulav
    May 23, 2017

    hello sir I am having problem in compiling the processing code..can you help me out...it shows some library are not installed properly...though I have already installed the serial and video library

  • Aswinth Raj's picture
    Aswinth Raj
    May 23, 2017

    This problem is peculiar. Double check if you have installed the libraries properly. If problem still exists, copy the error message from your procesing console and post it here that might help me to help you better.

     

  • drj jordan's picture
    drj jordan
    May 23, 2017

    pleased upload bluetooth based notice board.

  • Aswinth Raj's picture
    Aswinth Raj
    May 23, 2017

    Yes, Jordan will consider making one in near future. Stay tuned.....

  • sulav's picture
    sulav
    May 23, 2017

    how to do that sir? I am using HC-05 bluetooth module. please help me sir my deadline is very near.

  • sulav's picture
    sulav
    May 23, 2017

    type string[] of the last argument to method println(object) does not match the vararg parameter type. cast to object[] to confirm the non varargs innovation or pass individal arguments of type object for a varargs invocation.
    This is the error generated in setup function please help sir

  • Aswinth Raj's picture
    Aswinth Raj
    May 27, 2017

    The problem must be with the following two lines..

      video = new Capture(this, cameras[34]);

      port = new Serial(this,Serial.list()[1],9600);

     

    To solve this change the first line to 

      video = new Capture(this, cameras[0]);

    This will use the default cam on your laptop

     

     port = new Serial(this,Serial.list()[1],9600); To make this line work, you have to make sure that your laptop has a working Bluetooth and it is turned on and paired with your arduino. Open your Device manager and check if the Bluetooth has a valid COM port

     

  • sulav's picture
    sulav
    May 28, 2017

    yeah the same error is being generated please help sir

  • sulav's picture
    sulav
    May 31, 2017

    1) String[] cameras = Capture.list();
    2)println(Serial.list());
    The error is being generated on the above two lines. Please help sir

  • Sagar's picture
    Sagar
    Jan 27, 2018

    Sir can you plz help with the same error
    1) String[] cameras = Capture.list();
    2)println(Serial.list());

  • Aswinth Raj's picture
    Aswinth Raj
    Jun 02, 2017

    Hi sulav,

    I am not sure what might have caused the error. Kindly copy and paste the error message from the console to help you better.

    However, these two lines are the ones which access the cameras hardware information. Make sure your laptop has a functional webcam.  

  • abhay's picture
    abhay
    Jun 22, 2017

    hardware is working fine but the software end user application is not working. I'm getting just a black screen and grey square in the middle. please help me.

  • Aswinth Raj's picture
    Aswinth Raj
    Jun 22, 2017

    The problem occurs because your processing sketch is not abe to access the webcam of your computer.

    The problem must be with the following two lines..

      video = new Capture(this, cameras[34]);

      port = new Serial(this,Serial.list()[1],9600);

     

    To solve this change the first line to 

      video = new Capture(this, cameras[0]);

    This will use the default cam on your laptop

     

     port = new Serial(this,Serial.list()[1],9600); To make this line work, you have to make sure that your laptop has a working Bluetooth and it is turned on and paired with your arduino. Open your Device manager and check if the Bluetooth has a valid COM port

     

    Also to help you better copy the information on your console and paste it here

  • Ranjeeth's picture
    Ranjeeth
    Jun 28, 2017

    Sir could you please tell me what is the purpose of developing this system.
    Can i say it is an advancement of digital interactive board(SMART Boards) which is used in education institutions for lecturing purpose.

  • abhay's picture
    abhay
    Jun 30, 2017

    sir once after the hardware is set up what all should we do next? tell us step by step. please sir

  • uma maheshwari's picture
    uma maheshwari
    Jul 18, 2017

    sir, here we have seen the interfacing of our physical motion with a P C. Is it possible to interface our motion to some specific device in order to lift an object which is far away from us....If so kindly say a way as to what device can be used for interfacing and how to increase the range of operation.Is it possible to lift heavy weights too with ease.

  • Akash's picture
    Akash
    Aug 31, 2017

    hi....
    "No library found for processing.video"
    How to download processing.video..??
    plz help me

  • DAVID's picture
    DAVID
    Sep 04, 2017

    Hello sir greetings!
    Does this projects comes under sixth sense technology?

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Sep 07, 2017

    can i use any object to track and i m unable to understand the connection of tracking object plz explain it sir.

  • Aswinth Raj's picture
    Aswinth Raj
    Sep 08, 2017

    Yes you can track any object. The only limitation is that the colour of the object should be different than that of the background. 

    If you want to know more about how the tracking works. Please refer to the tutorial of Daniel Shiffman (link given in explanation). He will be able to add more value than I could

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Sep 24, 2017

    Is there any hardware connection between the tracking object and the controller?

  • Aswinth Raj's picture
    Aswinth Raj
    Sep 25, 2017

    No, there is not link between the object and controller. It is just a colored material placed in center so that processing could distinguish it. 

     

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Sep 24, 2017

    there is no link of Daniel Shiffman sir

  • sankar's picture
    sankar
    Sep 25, 2017

    The software and hardware part is fine. But still i can't able to calibrate hall sensor and I'm not getting the message as key 1 or key 2 is pressed. Help me to sort it out.

  • Aswinth Raj's picture
    Aswinth Raj
    Sep 25, 2017

    This problem is most likely due to the reason that you have connected to the wrong wrong COM port. Open you device manager and check to which port number your arduino is connected to. For example

     

    COM2 - Printer          0th port

    COM8 - Arduino        1st port

     

    In this case, the Arduino is connected to COM 8 but the port list number is 1 so "1" has to be entered in the processing sketch.

     

    You can check if your bluetooth is working fine by, using any other COM port monitor like Arduino Serial monitor before proceeding to Processing.

     

  • Aniket Mishra's picture
    Aniket Mishra
    Apr 23, 2018

    Hello sankar,did that solution solve your problem??
    we are facing the same problem..

  • Rodrigo's picture
    Rodrigo
    Oct 26, 2017

    How can I add more functionalities like change de color of the line or the the line thickness with more finger gestures? I'll be glad if you can help me out, it s for my digital electronics class proyect.

  • Aswinth Raj's picture
    Aswinth Raj
    Oct 26, 2017

    Hi Rodrigo,

    Yes everything you asked for is possible. You just have the change few lines of code in the processing skecth. Go ahead and fiddle around with it. If you have any doubts post them here or in the forum and I will help you out

  • Abhinandan garg's picture
    Abhinandan garg
    Nov 03, 2017

    Sir how to buy this tracking object. Plz suggest me

  • Aswinth Raj's picture
    Aswinth Raj
    Nov 06, 2017

    You don't have to buy anything like a tracking object. Just follow the instructions and above and build your own 

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Nov 06, 2017

    ok sir i will try...

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Nov 06, 2017

    can i replace arduino nano with arduino uno?

  • saad's picture
    saad
    Nov 16, 2017

    i want to ask you why we using hall sensor in these project What is the use of it??

  • Aswinth Raj's picture
    Aswinth Raj
    Nov 18, 2017

    The thumb has a magnet and the for finger and middle finger will have hall sensors. When you make the action of touching your fore finger with thumb to make a left click, the hall sensor will detect the magnet and instruct the Arduino that left click action is made

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Nov 28, 2017

    how to overcome the problem of " processing.video" does not exist. plz help me sir

  • AISHA's picture
    AISHA
    Nov 29, 2017

    It should be a  library problem. Did you install the video library for processing frst?

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Dec 03, 2017

    yes sir .

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Dec 03, 2017

    i installed all kind of library such as camera , video , sketch . but yet it show the error.
    it show the error in this line" String[] cameras = Capture.list(); "
    how to overcome this problem , tell me sir as soon as possible. bcz i have to submit this project in this week. so plz.....

  • AISHA's picture
    AISHA
    Dec 04, 2017

    This means that your Processing sketch is not able to access the camera of your computer. Did you try some basic camera related programs? Are you using windows?

  • Abhinandan garg's picture
    Abhinandan garg
    Dec 05, 2017

    Basic camera means?

  • Abhinandan garg's picture
    Abhinandan garg
    Dec 05, 2017

    Windows 10.
    It also show that this sketch should be run as a 32 bit application.

  • ABHINANDAN GARG's picture
    ABHINANDAN GARG
    Dec 05, 2017

    now it show the on led of camera when i use 32 bit application.
    but not showing any video through.
    it show me the error " [8] "name=USB2.0 VGA UVC WebCam,size=352x288,fps=20"
    [9] "name=USB2.0 VGA UVC WebCam,size=352x288,fps=30"
    Could not run the sketch (Target VM failed to initialize).
    when i use port = new Serial(this,Serial.list()[0],9600); this code .
    otherwise it shows " index value is out of boundary " when i use " port = new Serial(this,Serial.list()[1],9600); " now plz help me sir . time is too limited ....

  • abhiram's picture
    abhiram
    Dec 13, 2017

    hi,
    is there any specific reason why image processing is used to control the cursor movement(apart from accuracy),when the same could be achieved using accelerometer which would be much simpler.

  • Abhinandan garg's picture
    Abhinandan garg
    Dec 13, 2017

    Yes, it's very useful to provide the environment such as to paint and to drag the object virtually. U can see this in it's library also.

  • Aswinth Raj's picture
    Aswinth Raj
    Dec 14, 2017

    Hi Abhiram,

    Actually I have already tried to use an Accel. to control the mouse pointer

    https://circuitdigest.com/microcontroller-projects/ping-pong-game-using-...

    As you can see it works fine for one axis (X-axis). If we try to control both X and Y the accuracy is pathetic and also its hard to navigate by tilting the sensor.

    Also if you use image processing in a bit more advanced level, you can also completely eliminate the electrical hardware with user 

  • Syed farhan's picture
    Syed farhan
    Jan 09, 2018

    Sir !
    Can u please eleborate the making and connection more easily....

  • AISHA's picture
    AISHA
    Jan 11, 2018

    The circuit and code is already given!!!

    Why where r u facing problem? Give it a try and if you face problem use the forums

  • Akash's picture
    Akash
    Jan 27, 2018

    type string[] of the last argument to method println(object) does not match the vararg parameter type. cast to object[] to confirm the non varargs innovation or pass individal arguments of type object for a varargs invocation.
    This is the error generated in setup function
    And also the hall sensors are not detected .please help sir. No key 1/2 pressed message.

  • Satyam 's picture
    Satyam
    Feb 01, 2018

    Plzz help me .where to download libraries for processing...and how to install libraries

  • Deepak Malik's picture
    Deepak Malik
    Feb 02, 2018

    I am not able to see "Key-1 Pressed" or "Key-2 Pressed". I have entered correct COM port as well. I checked bluetooth module by sending the data through my phone and it is working properly. Application is also working properly but is not able to read the data from Bluetooth Module.

  • Aswinth Raj's picture
    Aswinth Raj
    Feb 05, 2018

    What is the name of your Bluetooth module? This code is written to recognize modules with name "HC-06".

  • Deepak Malik's picture
    Deepak Malik
    Feb 06, 2018

    We are using HC-05 module for the project. So what changes we have to do in our code?

  • AISHA's picture
    AISHA
    Feb 07, 2018

    Either change the name of your Bluetooth device as "HC-06" or change the processing code to detect HC-05 by replacing the word "HC-06" in the code with "HC-05"

  • Aniket's picture
    Aniket
    Feb 08, 2018

    Can we do this by not attaching the Arduino with the glove? Something like a Bluetooth connection between halo sensors and Arduino?

  • Abhinandan garg's picture
    Abhinandan garg
    Feb 14, 2018

    Yes u can easily test it without any attachment

  • Doris M Kumar's picture
    Doris M Kumar
    Feb 28, 2018

    Can you share the proper circuit diagram image ?

  • Aishwarya Karanth's picture
    Aishwarya Karanth
    Mar 16, 2018

    type string[] of the last argument to method println(object) does not match the vararg parameter type. cast to object[] to confirm the non varargs innovation or pass individal arguments of type object for a varargs invocation.
    Can someone please tell me how to debug it?

  • Abhinandan garg's picture
    Abhinandan garg
    Mar 20, 2018

    First of all import this library like this :
    import java.awt.event.KeyEvent;
    After that move on the line number 50 or 51 where is port . And replace it by this line
    port = new Serial( this, "COM3", 9600);
    COM3 can be COM5 in your laptop this can be different. You can check it from the Arduino software by following this step
    Tool> port
    Now remove next line in Processing software which is
    println( Serial. List());
    After that it will run without any error.

  • Aishwarya Karanth's picture
    Aishwarya Karanth
    Mar 20, 2018

    Where can i find the java.awt.event.KeyEvent; library? In order for the code to run, should Arduino and processing 3 be connected? If yes what is the procedure?

  • Aswinth Raj's picture
    Aswinth Raj
    Mar 22, 2018

    java.awt.event.KeyEvent; 

    This is not a library 

     

    No you need not connect Arudino and Procesing code. They will be connected through Bluetooth. Read the complete tutorial to know how it works

  • Aishwarya Karanth's picture
    Aishwarya Karanth
    Mar 20, 2018

    Will the code change for HC-05 bluetooth module?

  • Aswinth Raj's picture
    Aswinth Raj
    Mar 22, 2018

    Yes the code for pocessing will change. You just have to replace "HC-06" with "HC-05" in the processing code

  • Aishwarya Karanth's picture
    Aishwarya Karanth
    Mar 22, 2018

    "HC-06" is not explicitly used in the code. Where exactly should I make the changes?

  • Pratik Kadam's picture
    Pratik Kadam
    Mar 22, 2018

    Sir,I'm getting error.
    Displayed as 'Stray #'

  • GURWINDER's picture
    GURWINDER
    Mar 26, 2018

    I am getting this error. and not able run run the sketch..

    Type String[] of thr last argument to method println(object..) doesn't exactly match the vararg parameter type. cast to object[] to confirm the non varargs invocation, or pass individual arguments of type object for a varargs invocation

  • varun's picture
    varun
    Apr 06, 2018

    img.pixels[(Px+(Py*img.width))] = color(255); //color of the paint background updated......sir at this line its showing array index out of bounce exception while im drawing on paint sceen canu say me what to do to avoid it

  • Abhishek's picture
    Abhishek
    Apr 12, 2018

    Sir, i am not able to import the library to use video
    What do I do?

  • asil's picture
    asil
    Apr 18, 2018

    hi sir Im ask whats the objectives of this project and can u send me block diagram of this project

  • Aniket Mishra's picture
    Aniket Mishra
    Apr 21, 2018

    I have few questions.
    1. How many COM ports are shown in the running processing IDE?
    2. I am not able to calibrate hall sensor. Nothing appears on the screen.
    3. Bluetooth module connection is not stable.

  • Hiro_Hamada's picture
    Hiro_Hamada
    Apr 24, 2018

    1. The number of COM ports depends on your computer. It will be the exact number of the COM ports displayed in the device manager

     

    2. Check if the Bluetooth is working properly. Else it will not work

     

    3. What do you mean by not stable? It should work fine. 

     

    To rectify your problem connect the arduino directly to your PC thorugh the USB cable. Then check wich port your arduino is connected to and use the same COM number on Processing. This way you can bypass the HC-06 to check of things are wokring fine

  • Prasanna's picture
    Prasanna
    Apr 22, 2018

    As you said I tried to calibrate with blue object,I couldn't calibrate please help

  • Omkar's picture
    Omkar
    Apr 26, 2018

    any specific device name "HC-05" or "HC-06" is not mentioned in the processing code explicitly. How do we change the device name in the sketch?

  • Omkar's picture
    Omkar
    Apr 26, 2018

    Well, I managed to get past the "COM port busy" error. But now the application runs in a very small screen. Is it because it is in development mode? I have not made any other changes to the program.

  • Abhinandan garg's picture
    Abhinandan garg
    May 04, 2018

    Sir I completed all hardware set up but in the calibration it shows only Key 2 pressed when I pressed 2nd switch through magnet it doesn't show anything. And output in the processing is False, False rather then true.
    What can I do to fix it. Plz help me and I m using Windows 10

  • Keerthi's picture
    Keerthi
    May 04, 2018

    It says ArrayOutOfBoundException:34

  • abhishek's picture
    abhishek
    May 20, 2018

    sir,i am using the correct COM port but bluetooth light doesn't go stable and key pressed is not shown.

  • Jayant's picture
    Jayant
    May 22, 2018

    Did you pair your bluetooth module with laptop with the passkey? If that is done properly the blinking will stop 

  • ifraj's picture
    ifraj
    Jun 04, 2018

    i could't download your EXE file please help me out. plzzzzzzz

  • Aswinth Raj's picture
    Aswinth Raj
    Jun 06, 2018

    Hey I just tried downloading from the link and it works fine. Please try again you should be able to download 

  • Anuraj's picture
    Anuraj
    Jul 03, 2018

    Sir is it necessary to install youcam perfect? Because i didn't get any exe file for it.
    Pls help me sir

  • Hiro_Hamada's picture
    Hiro_Hamada
    Jul 16, 2018

    No, Youcam is not necessary. Just check if processing can gain access to your cam if yes then you cam is not needed

  • Anuraj's picture
    Anuraj
    Jul 16, 2018

    Sir could you help me in making a block diagram of its working?

Pages

Log in or register to post Comment