Feature Matching in Android

The feature matching algorithm in this tutorial is in JAVA with OpenCV4Android.

My purpose of writing the code is to find the perspective transformation between a pattern and an object in a scene. Basic steps to find a homography include 1) keypoint calculation 2) descriptor calculation 3) coarse matching 4) finer matching and 5) finding the transformation matrix. ( Let’s assume that the pattern has some nice features, and that we are dealing with gray-scale images )

First of all, we have to decide what feature to use for keypoint and descriptor calculation. It’s preferred to use SIFT or SURF features but they are not compiled with OpenCV4Android as they are non-free libraries. If you don’t want to separately compile those libraries, you could alternatively use FAST or ORB features; they also work decently. And for matching algorithm, Hamming distance/L1/L2 are usually preferred options.

FeatureDetector Orbdetector = FeatureDetector.create(FeatureDetector.ORB);
DescriptorExtractor OrbExtractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

For each of the two images (pattern and scene), we calculate its keypoints and descriptors

Mat descriptors1 = new Mat();
MatOfKeyPoint keypoints1 = new MatOfKeyPoint(); 
Orbdetector.detect(image1, keypoints1);
OrbExtractor.compute(image1, keypoints1, descriptors1);

Then we do matching between the two sets of descriptors of the two images. Before this step, it’s better to check the size of descriptor1 and descriptor2 to see if they have enough points

MatOfDMatch matches = new MatOfDMatch();
matcher.match(descriptors1,descriptors2,matches);

We want to discard some of the poor matches, and thus we set a threshold and only store matches that have a distance below the threshold

 LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
 MatOfDMatch gm = new MatOfDMatch();
 for (int i=0;i<descriptors2.rows();i++){
   if(matchesList.get(i).distance<3*min_dist){ // 3*min_dist is my threshold here
     good_matches.addLast(matchesList.get(i));
   }
 }
 gm.fromList(good_matches);

Once we got a list of good matches, we can extract these pairs of points from the two images. If you want to match image1 to image2 (or find the transformation matrix from image1 to image2), then trainIdx is used for image1 while queryIdx is used for image2

LinkedList<Point> objList = new LinkedList<Point>();
LinkedList<Point> sceneList = new LinkedList<Point>();
for(int i=0;i<good_matches.size();i++){
  objList.addLast(keypoints2_List.get(good_matches.get(i).trainIdx).pt);
  sceneList.addLast(keypoints1_List.get(good_matches.get(i).queryIdx).pt);
}

The inputs to the function “getPerspectiveTransform” have to be in Mat format, convert the linked list to Mat formats and then find the transformation matrix between the two sets of good matching features

obj.fromList(objList);
scene.fromList(sceneList);
Mat perspectiveTransform = Imgproc.getPerspectiveTransform(obj,scene);
Advertisements

[Installation]Installing OpenCV 2.4.10 on Yosemite

Newest version of OpenCV 2.4.10 is out and ready for download!

Check if Xcode is installed

First check if Xcode is installed on your machine. To do that, open a Terminal and type:

$ xcode-select -p

This gives you the path to your installed xcode. If you see

$ /Applications/Xcode.app/Contents/Developer/

Your xcode is fully installed! You can also check the version of your xcode by typing:

$ /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild -version

Download a Package Manager

Multiple package managers do the job. (e.g. Macports, Fink or Homebrew). In my case I used Macports, so everything below is based on Macports. You can check to see if it installed successfully by opening your terminal and typing:

$ port

Use Macports to get cmake

In your terminal, type in the following:

$ sudo port install cmake

This will go fetch cmake and its dependencies and install them onto your system. You can check to see if cmake is installed by typing

$ cmake

Download and Build OpenCV

Sometimes opencv.org just failed to load. I downloaded the latest OpenCV from SourceForge. (They update it so you always get the newest version!)

Download the .zip file and unzip it to a directory. We are going to build OpenCV using cmake. In terminal, go to the directory where OpenCV was extracted to. Type in the following to make a separate directory for building purpose:

mkdir build
cd build
cmake -G "Unix Makefiles" ..

Notice that there is a space before the two ending dots.

Now, we can make OpenCV. Type the following in:

make -j8
sudo make install

This should now build OpenCV into your /usr/local/ directory.

If you want to double check to ensure it is installed, go to the directory /usr/local/lib (to do this, click the ‘Go’ tab at the top of your screen and click ‘Go to folder’ to type in the directory). Clean up the files by ‘Date Modified’, you can see those **2.4.10.dylib files.

Congratulations! Now you can do a bunch of super duper cool stuff with this awesome vision tool.