Assignment 3

• Sample Images

  Can you post the sample images provided in the handout?

  Indeed. This zipfile includes .bmp's of the four sample images.

• OpenCV flow issues

  The dense LK flow function included with OpenCV seems to give bad results? What gives?

  It does seem that for many images and for large flows, the OpenCV dense flow function provides [very small / very large / inconsistent / random] flows. You might want to follow the OpenCV flow example, which uses the feature-based pyramidal flow function. Plus it's always nice to work from an example.

• Matlab and OpenCV

  Can we use OpenCV for some parts of this assignment and Matlab for others?

  Yes... you can call C++ functions from Matlab or vice versa. If it's way easier for you to actually run some steps in Matlab and write out files to be read into a C++ program, or vice versa, that's fine too... just make it very clear what steps we need to take to run your whole package on your images.

Assignment 2

• Matlab Performance

  Matlab is running hella slow... any tips on speeding up Matlab programs?

  Our first reminder is that you won't be graded on efficiency, so don't stress too much about this. With that said, if you're into optimization, the image below was generated in under five seconds on a 3GHz desktop PC.

  The tips I'm about to give you basically say "pre-allocate" and "never loop"... in fact you can pre-allocate everything in this assignment and have no nested loops, and in fact the only loop you need can be way smaller than either dimension of your image. But that's only if you're feeling ambitious... :)

  The two most important tips for speeding up anything in Matlab are:

  • Pre-allocate all of your matrices... i.e. never _append_ data to a matrix or vector. You can use the zeros(m,n) command to create an empty matrix. Want to see a really slow Matlab program?

    a = 0;
    for(i=1:1000000)
      a(i) = 4;
    end
    

    This takes... well I didn't have time to let it finish. A better approach to this totally useless operation would be:

    a = ones(1,1000000) * 4;

    This runs in 0.031 seconds.

  • Anything that's in a small for loop should be vectorized... for example, if I'm taking the squared difference every element in two matrices A and B, (an arbitrary example, unrelated of course to hw2, of course), I could do this:

    
    difference = 0;
    for(i=1:blah)
      for(j=1:blah)
        difference = difference + (A(i,j) - B(i,j) ) .^ 2;
      end
    end
    

    ...but it would be much better to do this:

    difference = sum(sum((A-B) .^2));

    The natural extension of that is to turn large multiplications into convolutions, which makes your code hella fast but slightly harder to read, so save it until you need it.

  • Also, make sure you're not searching parts of the image you don't need to... i.e, pixel (i,j) in one image can only correspond to... hmmm...

  Disparity Mapping

  Can we assume that we only need to search along the horizontal axis for correlations?

  Yes.

  Are we graded on the efficiency of our solution?

  We do need to be able to run your code, so it should run in a "reasonable" amount of time (ten minutes per image is not reasonable, for example). But within that constraint, you will not be graded on efficiency. So you shouldn't really have to spend any time on speed optimizations, unless optimizing your Matlab code gets you fired up on an emotional level.

  What does a good disparity map look like?

  One obvious one is that the anaglyph (the magic-eye image) should give you an obvious result.

  For the 'real' images, you should get a relatively smooth field that gets darker as the road gets farther away and has blobs where objects are. Here's a good example for image 7, generated with no fancy optimizations (hence all the gunk at the top and bottom of the image) (click for a larger image)...

• Question 2.1

  It will make life easier for the staff if everyone in the class uses the same intrinsic parameters... here are some good values:

  • f/s_x: 462.7, f/s_y: 461.2, resolution: 352x288
  • Hint: we wouldn't be telling you this if these numbers weren't useful for solving this problem
  • Hint: ...but this is already more numbers than you need...

  Some other assumptions that you can make:

  • Don't worry about noise or distortion, assume you have perfect cameras that can reliably detect a one-pixel disparity.
  • Assume the object is directly in front of one of the cameras; the location directly in front of the baseline center is an annoying special case.

  If you already did this problem using alternative assumptions, let us know in your submission and clarify your assumptions.

Assignment 1

• Corner finding:
  Only images where all corners are found by cvFindChessboardCornerGuesses are valid. If not all corners are found, the function does not provide proper ordering of the corners. Don't try to fix this by manually ordering the points, just take a few better pictures.
• Data types:
  The types CvVect32f etc are not properly defined in the documentation. Look into openCV/cv/include/cvtypes.h instead. Also, don't mix up CvVect32f and CvVect64d or you will get very weired results. For the assignment, it is easiest to use only the 32f version (i.e. floats).
• Visualization:
  Printing and sending the TA a long list of point correspondences does not help, we just don't have the time to verify hundreds of points per group. Also, this is a very error-prone method. Instead, try a proper visualization, for example putting a circle around every corner in the image and its number next to that.
• 3D scene corner locations:
  Yes, you can re-use the 3D coordinates on the chessboard for every image, you do not need to change them for every image. Aside from estimating the camera calibration values, cvCalibrateCamera also estimates the rotation and translation of the chessboard towards the camera, and that for every image.
• Point format:
  To store the 2D and 3D Points for multiple images, don't use pointers or any sophisticated data structure. All points should be in 1D arrays of the cvPoint2D/3D32f data type. The first n points are for the first image, the next for the second and so on.
• How many images and corners?
  Although in theory, very few corners and images are necessary to calibrate the camera (as Sebastian proved during the lecture), in practice you should use something like at least 10 images of at least a 5x6 chessboard to get proper results.

Projects FAQ

• Should I perform a self-defined project?
  Unless you know well what you want to do as a project and a at least have a rough idea of how you want to pursue that project, we would not recommend taking a self-defined project. The reason for that is that it is likely that your project will be either too easy/small or too hard/large for the scope of this class. But of course, we don't want to discourage you from thinking about cool projects. If you are confident you have something intersting, please write up a short project description in the same format as the ones already on this page and e-mail it to cs223b@gmail.com.