Underwater Buoy detection using Gaussian Mixture Models (GMM) and Expectation-Maximization (EM) Algorithm

This involves implementing the concept of color segmentation using Gaussian Mixture Models and Expectation Maximization techniques.

• The input video sequence data has been captured underwater and shows three buoys of different colors, namely yellow, orange and green. They are almost circular in shape and are distinctly colored.
• Conventional segmentation techniques involving color thresholding will not work well in such an environment, since noise and varying light intensities will render any hard-coded thresholds ineffective. In such a scenario, the color distributions of the buoys is learnt and that learned model is used to segment them.
• A tight segmentation of each buoy for the entire video sequence has been obtained by applying a tight contour (in the respective color of the buoy being segmented) around each buoy.

Approach and implementation:

• The data preparation phase involved computation and visualization of the average color histogram for each channel of the sampled RGB images for each colored buoy separately. This provides some intuition on how many Gaussians (N) are required to fit to the color histogram.
• A 1-D Gaussian is used to model the color distribution of the buoys. Segmentation of the colored buoys was tried using this 1-D gaussian.
• Implementation of Expectation-Maximization algorithm was done and can be found in sample_em_learning.py.
• EM algorithm was used to compute the model parameters, i.e. the means and variances of the N 1-D Gaussians.
• Given the computed model parameters a color-segmented binary image was generated from the frames of the video sequence.
• Elaborate explanation about the approach and the pipeline can be found in the report

Output:

Buoy Detection for each individual buoy along with its binary image:

All buoys detected:

Output videos:

• Buoy detection/segmentation using single gaussian.
• Buoy detection/segmentation using multiple gaussians, here 3 for all buoys.

Instructions to run the code:

Input video dataset: here or here.

• To get the avghistogram: run:

  • $ python avg_histogram.py r or

  • $ python avg_histogram.py g or

  • $ python avg_histogram.py y

• To train the models:

  • $ python buoytraining.py r 1 or

  • $ python buoytraining.py r 3 or

  • $ python buoytraining.py g 1 or

  • $ python buoytraining.py g 3

  • First argument is for buoy color and second is for number of gaussians to fit.

• Model params are there in the folder named 'modelparams'.

• To run the buoy detection script:

  • $ python segment_buoy.py 1 1 1 or

  • $ python segment_buoy.py 3 3 3

  • here, 1 1 1 or 3 3 3 represent the number of gaussians there for red, green and yellow buoy respectively.