Installation
Just a quick note. There are plenty of guides on OpenCV installation. Checking out the sources and building with CMake is a breeze. It is also required for CUDA support.Sources
All sources for this are on my github.Preprocessing Steps
We do the following preprocessing on each image:- Find eye contours, create mask
- Color transfer via histogram specification
- Convert to HSV, apply CLAHE on value channel
- Convert back to BGR (this OpenCV, so not RGB, but BGR), apply mask to filter out the background
Data Structures, Memory Management
Unlike Python, where OpenCV images are stored in NumPy arrays, in C++ OpenCV 2.4 uses Mat and GpuMat. Some algorithms work on GPU, some don’t. That means memory moves between RAM and GPU memory may become an issue, since it is one of the more time consuming operations in GPU development. Straight CUDA code for memory moves is cumbersome, but not a problem in OpenCV. On the other hand, when wrapping all kinds of transforms in a class, trying to keep track of where the current object on which a transform should be performed resides, may become complicated and hard to maintain. My first stab at this was, perhaps, not the most elegant:class TransformImage{protected:
Mat _image; //original
Mat _enhanced; //after all transforms
Mat _buf; //buffer for intermediate operations
gpu::GpuMat g_image;
gpu::GpuMat g_enhanced;
gpu::GpuMat g_buf;
inline void MakeSafe() { g_enhanced.copyTo(g_buf); }
...
}
void setImage(Mat& image)
{ image.copyTo(_image); g_image.upload(image); g_image.copyTo(g_enhanced); }
Mat& getImage() { return _image; }
void setChannel(Channels channel) { _channel = channel; }
Mat& getEnhanced()
{ g_enhanced.download(_enhanced); return _enhanced; }
CLAHE on GPU
For a discussion of histogram equalization, see this tutorial.
This is sorta out-of-band for this post, but to dilute all this book-keeping, I thought it would be good to demonstrate at least how to do CLAHE on the GPU. The code does not differ much from a CPU code!
My transform class applies the CLAHE algorithm to a single channel of the image. I have g_oneChannel array that holds different channels of the image. CLAHE is designed to work on one channel (grey scale) images. If we want to apply it to an RGB (or, in the case of OpenCV BGR) images, we first need to convert the image to HSV (or HSI or some such), apply the algorithm on the value (intensity) channel, and merge that channel back into the original image. This is where the channel stuff in my C++ code comes from. Fortunately we can do all of this without leaving the GPU.
gpu::GpuMat& ApplyClahe()
{Ptr<gpu::CLAHE> clahe = gpu::createCLAHE();
clahe->setClipLimit(4.);
clahe->setTilesGridSize(Size(16, 16));
clahe->apply(g_oneChannel[(int)_channel], g_buf);
g_buf.copyTo(g_oneChannel[(int)_channel]);
gpu::merge(g_oneChannel, g_enhanced);
return g_enhanced;
}
Notice the use of g_buf to hold an intermediate result to be safe. Not sure if we could have skipped it, I didn’t check.
This code is reminiscent of the Python code (as the tutorial shows):
import numpy as np
import cv2# create a CLAHE object (Arguments are optional).clahe = cv2.createCLAHE(clipLimit=2.0, tileGridSize=(8,8))
cl1 = clahe.apply(img)
return cl1
Here are the “before” and “after” CLAHE images of Kaggle 16_left.jpeg
 |  |
Hello, I was wondering if you know how to use gpu::pyrDown and gpu::pyrUp in opencv? I’ve been having a very hard time finding anything related to my problem and I was wondering if you could help.
for (int i = 0; i < Pyramid_Size; i++) {
cv::gpu::pyrDown(DownUp, DownUp);
}
for (int i = 0; i < Pyramid_Size; i++){
cv::gpu::pyrUp(DownUp, DownUp);
}
Is where the problem lies. I found that once I use pyrDown and pyrUp each pixel is at a value of 0 inside GpuMat DownUp. Do you know why this may be?
I think the problem is you cannot have your source and destination images refer to the same location. Take a look here:
https://github.com/fierval/retina/blob/master/FeatureExtractionCpp/TransformImage.hpp
gpu::GpuMat& PyramidDown(int scale = 1)
{
gpu::GpuMat buffer;
g_image.copyTo(buffer);
for (int i = scale; i > 0; i--, g_enhanced.copyTo(buffer))
{
gpu::pyrDown(buffer, g_enhanced);
}
return g_enhanced;
}