Integral channel feature


Integral Channel Features, also known as ChnFtrs, is a method for object detection in computer vision. It uses integral images to extract features such as local sums, histograms and Haar-like features from multiple registered image channels. This method was highly exploited by Dollár et al. in their work for pedestrian detection, that was first described at the BMVC in 2009.

Overview of the method

  1. Compute multiple registered image channels from an input image, using linear and non-linear transformations
  2. Extract features such as sums over rectangular channel regions from each channel. The features extracted from various channels are called integral channel features.
  3. Train the AdaBoost classifier. Dollár et al. used boosting technique which offers faster learning but training could be done with any of the other available methods such as support vector machine.
  4. Finally, trained classifier is used to detect objects

    Images and channels

Typically, a "channel" refers to a certain component that defines pixel values in a digital image. A color image, for example is an aggregate of three channels. The color data of an image is stored in three arrays of values, known as channels. While this definition of a "channel" is widely accepted across various domains, there exists a broader definition in computer vision, which allows one to exploit other features of an image besides the color information. One such definition refers to a channel as a registered map of the original image where the output pixels are mapped to input pixels by some linear or non-transformation. According to this notion of a channel, color channels of an image can be redefined as output images that are obtained by extracting one specific color information point from the input image at a time. Similarly, a channel for a grayscale input image is simply equal to a grayscale input image. The simple MATLAB implementation below shows how color channels and grayscale channel can be extracted from an input image.

I = imread; %input color image
%Output_image = color_channel,
%where color channel could be red, green or blue. The three output images
%are extracted from input image as follows
red_channel = I;
green_channel = I;
blue_channel = I;
%Output image = grayscale_image.
%Note if input image I was already a grayscale image, grayscale channel
%would have simply been equal to input image, ie, gray channel = I
gray_channel = rgb2gray;

It is clear from the above examples that a channel can be generated by either simply extracting specific information from the original image or by manipulating the input image in some form to obtain the desired channel. Dollár et al. defined a channel generation function as Ω, which can be used to relate a channel to the original image as follows.
The next section discusses other relatively complex channel types as mentioned in the original paper by Dollár et al. MATLAB implementation is given for some of the channels.

Various channel types


%Output image = DoG
%Difference of Gaussian applied on input image
H1 = fspecial; %create a Gaussian with signal 0.5
H2 = fspecial;%create a Gaussian with signal 3
DoG_filter = H1-H2; %create a DoG
image = double;
DoG_channel = conv2;%convolve DoG with input image

Note that these channels can be used alone or in combination with each other.

Feature extraction

Once channels are obtained from an input image, various features can be extracted from these channels. These features are called channel features and can be categorized into two main types:
The ChnFtrs method allows one to pool features that capture the richness from diverse channels. Dollár, et al. based their experimental results on first order features since there was not much added value by the second order features. The channels are re-computed at multiple scales to extract a pool of channel features that are can represent the entire scale space.
There is a MATLAB toolbox that can be used as a guidance to implement ChnFtrs method. Further, OpenCV has a complete implementation of ChnFtrs.

Performance

To study the performance of ChnFtrs, Dollár et al. first evaluated the effectiveness of various channels when used individually. The channels studied were histogram of oriented gradients, gradient histogram channel, gradient magnitude, color channels and grayscale channel. The performance was evaluated in terms of pedestrian detection rates at the reference point of 10 - 4 fppw. HOG turned out to be the most informative channel compared with rest of the channels. The detection rate of HOG was 89%. Further, among the color channels, LUV had the best detection rate of 55.8%. Grayscale channel was least informative with the detection rate of only 30.7%.
Next, they evaluated the performance of various channel combinations, which is their proposed method. The combination of LUV, Hist and Grad channels had the highest detection rate of 91.9%. This channel combination was further used in their experiments on INRIA and Caltech datasets.
About 30,000 first order features were used to train AdaBoost classifier. The ChnFtrs + AdaBoost detector was tested on full images from INRIA and Caltech datasets. The performance was compared with 12 other detectors including HOG, which is the most popular method. ChnFtrs outperformed all except LatSvm. The detection rate for ChnFtrs was 86% on INRIA dataset and 60% on a more challenging Caltech dataset.

Further development

The ICF method has been widely exploited by researchers in Computer Vision after the work was initially published by Dollar et al.. In fact, it is now used as a baseline detector due to its proven efficiency and reasonable performance. Several authors have obtained even better performance by either extending feature pool in various ways or by carefully choosing the classifier and training it with a larger dataset. Work by Zhang et al also exploited integral channel features in developing Informed Haar detector for pedestrian detection. They used the same combination of channels as Dollár et al. but were able achieve approximately 20% higher performance than the baseline ChnFtrs method. The added performance was due to the fact that they provided better prior knowledge to their detector. It is also important to note that they used informed Haar-like features, which are second order features according to the terminology described in, whereas Dollár et al. demonstrated their results using first order channel features only, as their analysis showed that second order features barely added 0.6% increase to their detection rate. Further, Benenson et al. were able to increase the detection speed of baseline ChnFtrs method by avoiding the need to resize input image.
ChnFtrs is a versatile method that allows one to extract features from multiple channels, thus allowing to capture diverse information from a single input image. The performance of a base detector developed by Dollár et al. has been shown to be enhanced by adding better prior knowledge and training with a larger dataset.