Image Region Classification based on Multiclass SVM

- Project Milestone

Lei Liu

Goal

The goal of the project to build an image region classification tool that is capable of classifying each pixel in an image into five different categories using Multi-class SVM:

Sky: sky including clouds

Ground: road, sand, …

Water: sea, river, lake, …

Building: houses, ...

Vegetation: grass, tree

Done

1. Data Collection

MSRC-V2 Image Database: publicly available "semantic segmentation" image databases which contain 591 photos manually segmented into 23 object classes, including the five classes we aim to classify.

Steps:

- Get a set of tiles (we choose a tile to be a square subdivision of the image with length of 15 pixels) for each class from the image database: For each image in the database, subdivide it into tiles and store tiles that are entirely inside one of the five target regions into corresponding groups.

- For each class, randomly select 1500 tiles as training set and another 1500 tiles as testing set.

2. Feature Extraction

We need to compute a low-level description of tiles before submitting them to the classifier. Since the experiment with HSV histogram alone didn't give good results, I decide to use joint histogram which combines color distribution with gradient statistics. Color distribution is represented by the HSV color space and the vertical and horizontal gradients are computed by applying Sobel's filter to the luminance image.

Partition of HSV color space^[1] Partition of vertical and horizontal gradients^[1]

Since HSV color space is partitioned into 11 bins and both vertical and horizontal gradients are partitioned into 4 bins, the total number of bins for the joint histogram is 11*4*4 = 176.

3. Multi-class SVM algorithm

The soft margin Gaussian-kernel SVM algorithm is implemented using the built-in Matlab quadprog function. Multi-class SVM algorithm is constructed according to One-versus-the-rest strategy. Thus in total we train five SVM classifiers and each was trained to discriminate between one class and the others.

To train the SVM classifier, we use the 1500 tiles of the related class and a random selection of 1500 tiles form the other classes. To test the SVM classifier, we use all the 5*1500 = 7500 tiles from the test set to evaluate its performance.

4. Results

Test all possible combination of Guassian Kernel parameter Sigma = [0.1, 0.5, 1.0] and C = [5, 25, 50]

The combination of [1.0, 50] gives the best correct rates as following:

Building: 52.9%

Ground: 82.2%

Sky: 87.3%

Water: 82.1%

Vegetation: 82.7%

One possible reason for the poor performance of Building classifier is the large distribution of both color space and gradients in building images.

Todo

- Improve classifier performance, especially building classifier

1. Enlarge the training set for Building classifier, I’ve tried this on my laptop but Matlab complained that 2GM memory is not large enough, so I may need to export

the computation to a server with enough memory

2. Test more combinations of Sigma and C, use cross-validation to choose the best one

3. Try different tile sizes

- Currently the training is very slow because Matlab quadprog uses primal form, I need to try to solve the dual form instead

- Pixel-wise Classification: For each pixel, each tile that contains it is classified independently and the pixel's final classification is determined by majority vote

References:

[1] Claudio Cusano, Gianluigi Ciocca, Raimondo Schettini, etc. "Image Annotation using SVM". Proceedings of the SPIE, Volume 5304, pp. 330-338, 2003.
[2] C.Faloutsos, R.Barber, M.Flickner, etc. "Efficient and effective querying by image content". Journal of Intelligent Information Systems, 3, pp. 231-262, 1994.
[3] MSRC-v2 image database, http://research.microsoft.com/en-us/projects/objectclassrecognition/ .
[4] M. Stricker, and M. Swain, "The Capacity of Color Histogram Indexing", Computer Vision and Pattern Recognition, pp. 704-708, 1994.