Wednesday, February 15, 2012

OpenCV Stereo Calibration

Whenever you play with stereo computation, Stereo Calibration plays a crucial step. In this post, I will explain what is Stereo Calibration and what are steps to follow for performing Stereo calibration using  OpenCV.

Why do I need to perform Stereo Calibration?

Stereo cameras can be mainly used in robotics. Calibration of camera is vital to overcome distortions introduced by lens and other misalignment, so that it can be used for further processing. OpenCV makes use of algorithm to perform this stereo camera calibration using standard test images like chessboard or dots. It computes extrinsic parameters (rotational and translational matrix of stereo camera) and intrinsic parameters (focal length, baseline and optic center).


What You need for Stereo Calibration?
  1. Stereo Camera, it may be your custom made two USB camera or two web cams or from professional stereo camera manufacturer.
  2. Print out of chessboard. You can get chessboard from [OpenCV Install Dir]\doc\pattern.png. Don't worry about clarity or resolution problem. It will be of higher resolution and no worries:-)
  3. Once you take print out, paste it on rigid surface. You can use acrylic sheet or card board. 
How to capture chessboard Images for Stereo Calibration?

If you don't have any application to capture chessboard images simultaneously from left and right camera, here is code you can put and compile.

#include "stdafx.h"

//OpenCV specific Headers
#include "cv.h"
#include "highgui.h"

int main()
{
 IplImage *frameL = 0;
 IplImage *frameR = 0;

 CvCapture *captureL = 0;
 CvCapture *captureR = 0;

 int key = 0;
 int iFileName = 0;
 char FileName[MAX_PATH] = "";

 //Initialize left and right camera
 captureL = cvCaptureFromCAM(0);
 captureR = cvCaptureFromCAM(1);

 //Always check for valid frame capture.
 if (!(captureL && captureR))
 {
  printf("\nCannot open initialize webcam!\n" );
  
  //Release CAM Handle.
  cvReleaseCapture(&captureL);
  cvReleaseCapture(&captureR);
  return 0;
 }

 printf("\nPress 's' to capturing image pairs.\nPress 'q' to quit application\n");

 while( key != 'q' ) 
 {
  //Get a frame
  frameL = cvQueryFrame(captureL);
  frameR = cvQueryFrame(captureR);

  //Check for frames
  if(!(frameL && frameR))
   break;

  //Display current frame
  cvShowImage("Left", frameL);
  cvShowImage("Right", frameR);

  //Exit if user press 'q', and save frame if pressed 's'
  key = cvWaitKey(1);


  if(key == 's')
  {
   //Save Left CAM Image.
   sprintf(FileName, "Left%d.jpg",iFileName);
   cvSaveImage(FileName, frameL);

   //Save Right CAM Image.
   sprintf(FileName, "Right%d.jpg",iFileName);
   cvSaveImage(FileName, frameR);

   //Lets update in console on how many images been captured till now.
   printf("\n %d Pairs Captured", iFileName+1);
   iFileName++;
  }
 }

 //Always perform cleanup.
 cvReleaseCapture(&captureL);
 cvReleaseCapture(&captureR);
 return 0;
}

Before compiling this code, dont forget to add OpenCV dependency libraries to Visual Studio project properties. Check how to add opencv dependency library post to know how to do this. The above code will let you to capture and save images from left and right camera in your working directory. Once completed capturing with required number of chessboard images, next steps is to perform stereo calibration. The below image is screenshot of the above code.



How to get source for OpenCV Stereo calibration?

Source code for performing OpenCV Stereo calibration can be found in [OpenCV Install Dir]\samples\cpp\stereo_Calib.cpp. Following are inputs needed for performing stereo calibration using this application.

  1. Number of horizontal corners.
  2. Number if vertical corners.
  3. Chessboard square size in cm.
For chessboard given in OpenCV documents,
number of horizontal corners = 9
number of vertical corners = 6.
Square size depends on paper on which size paper you are taking print out. For A4 size paper, square size will be of ~2.7cm and for A3 size paper it will be ~5.5cm. Its better for you to measure using ruler before providing input for calibration. 

Once stereo calibration is completed, it will save bunch of stereo calibration results as either xml or yml based on what you coded. Stereo Calibration results can be used for remapping and distance computation.
Remapping is used for stereo rectification, where curvature effect due to lens will be rectified. I will tell you on how to perform stereo correspondence to find disparity map of stereo images in my next post:-) Keep following to be updated.

4 comments:

  1. thanks Sathya it was really very helpful...I have one query ..for my project I need the focal length,baseline distance and the optic center..will this all save in the XML file?

    ReplyDelete
    Replies
    1. Welcome MMH. Sorry, was busy in analyzing other part of OpenCV related libraries. Yup, all these will be saved in Q.xml or what ever name you use. Focal length will be located at (2,3), reciprocal of baseline in centimeter will be at (3,2) and difference in optic center multiplied by baseline is located at (3,3).

      You can use this for depth measurement.

      Delete
  2. thanks for this satya.
    i'm using linux. i've commented out "#include windows.h". i've replaced MAX_PATH with 500. when i run it i get the two images but ehwn i press either "s" or "q" nothing happens.
    also there is a delay between the two frames updating can this be lessened or eliminated?
    thanks.

    ReplyDelete
    Replies
    1. Hi Jaysin,

      Let I split and answer your questions.

      1. Yes, you will get two images, 1 shows the streaming from left camera and another from right camera. cvShowImage() is used for displaying image received from camera.

      2. Make sure, your console window is active while pressing key.

      3. We cannot optimize delay between received frames at application side. Optimization can be done only by making modifications on OpenCV library, which is a time consuming and difficult process. Also make sure, your camera is running at full frame rate.

      Regards,
      Sathya Kumar P

      Delete