Building a Linear Gaussian Classifier to Classify Cat and Grass

in Classifier

This project develop a cat-grass two class classifier based on maximum-a-posteriori model (no neural network or deep learning). The entire classifier is implemented in python using numpy module mainly. The result accuracy for the test image is over 90% but it does not generalize to other images.

This blog shows the steps needed to build a Gaussian classifier using pre-computed data to classify whether a pixel in an image belongs to a cat or grass. The goal of the project is to show a use case of the maximum-a-posteriori model of a simple classification task. This does not involve any neural network or deep learning. The entire project is implemented using python.

This project and all associated data are provided by Prof. Stanley Chan from Purdue University for learning purposes only.

Part 1 - Setting Up

This project is done in python. I’m using python version 3.7 but other versions of python should also work. In the case of python 2.x, there might be some differences in syntax. Python installation instruction can be found here.

The following two packages are also used in this project: numpy and matplotlib. Those can be installed by running:

pip install numpy
pip install matplotlib

The entire data packages, including the images and raw data needed for building the classifier, can be downloaded here. The zip file contains:

  1. cat_grass.jpg: the input image to be classified;
  2. truth.png: the ground truth of the classification;
  3. train_cat.txt: the training data for the cat model;
  4. train_grass.txt: the training data for the grass model.

The use of those four files will be explained later.

Part 2 - Theory

Both train_cat.txt and train_grass.txt can be interpreted as raw csv files with 64 rows and n columns. Each column represents the pixel value for an 8x8 patch in the image that corresponds to each class. The theory here is that those patches from each class (cat or grass) behave like a Gaussian model with some mean and standard deviation:

\pmb{\mu}_{(\text{class})}=\frac{1}{K}\sum_{k=1}^{K}\pmb{x}_{k}^{(\text{class})}

and

\pmb{\Sigma}_{(\text{class})}=\frac{1}{K}\sum_{k=1}^{K}\left(\pmb{x}_{k}^{(\text{class})}-\pmb{\mu}_{(\text{class})}\right)^T\left(\pmb{x}_{k}^{(\text{class})}-\pmb{\mu}_{\text{class}}\right)

where K is the total number of training examples (number of numbers per row in the given file). Based on this model, assume that an unknown patch is from certain class, the probability of seeing that patch \pmb{z} can be modeled as conditional probability:

f_{\pmb{Z}|\text{class}}\left(\pmb{z}|\text{class i}\right)=\frac{1}{(2\pi)^{d/2}\left|\pmb{\Sigma}_{\text{class i}}\right|^{1/2}}\exp\left\{-\frac{1}{2}\left(\pmb{z}-\pmb{\mu}_{\text{class i}}\right)^T\pmb{\Sigma}_{\text{class i}}^{-1}\left(\pmb{z}-\pmb{\mu}_{\text{class i}}\right)\right\}

By the Bayes’ Theorem, the probability of a patch \pmb{z} falls under certain class is:

f_{\text{class}|\pmb{Z}}\left(\text{class i}|\pmb{z}\right)=\frac{f_{\pmb{z}|\text{class}}\left(\pmb{z}|\text{class i}\right)f\left(\text{class}\right)}{f_{\pmb{Z}}(\pmb{z})}

where f_{\text{class}}\left(\text{class}\right) is our prior knowledge about the problem: without looking at what the patch look like, what is the probability that it is from a certain class. In our example, this can be seen as the ratio of total training example that belongs to that class:

f_{\pmb{Z}|\text{class}}\left(\text{class k}\right)=\frac{K^{\text{(class k)}}}{\sum_{\text{classes}} K}

In our case will simply be the ratio between cat training samples to the total training samples and grass training samples to the total training samples. For our classifier, there are two possible classes: cat or grass; therefore we are comparing f_{\text{class}\vert{\pmb{Z}}}\left(\text{cat}\vert\pmb{z}\right) vs. f_{\text{class}\vert{\pmb{Z}}}\left(\text{grass}\vert\pmb{z}\right). We can make conclusion as following:

\pmb{z}=\begin{cases}\text{cat}&\text{if\ }f_{\pmb{Z}|\text{class}}\left(\pmb{z}|\text{cat}\right)f\left(\text{cat}\right)>f_{\pmb{Z}|\text{class}}\left(\pmb{z}|\text{grass}\right)f\left(\text{grass}\right)\\\text{grass}&\text{if\ }f_{\pmb{Z}|\text{class}}\left(\pmb{z}|\text{cat}\right)f\left(\text{cat}\right)<f_{\pmb{Z}|\text{class}}\left(\pmb{z}|\text{grass}\right)f\left(\text{grass}\right)\end{cases}

This decision rule based on the above equation is called Maximum-a-Posteriori (MAP) decision.

Part 3 - Building the Classifier

This is a long procedure. I’m breaking it down into separate steps.

Part 3.0 - Import

Before implementing the classifier, two installed libraries are needed. We will also import the built-in module time to have a rough idea of how long does the program takes.

import numpy as np
import matplotlib.pyplot as plt
import time

Part 3.1 - Load the files

We can load the images using the matplotlib library and the training texts using numpy library. When the jpg image is loaded the default value will be from 0 to 255. We have to rescale it to [0, 1] instead. This does not apply to the png file. For the ground truth file, we have to round each value to 0 or 1 before using it. The image contains values like 0.1 for black pixels and 0.8 for white pixels. All that we care about is whether it is white (1) or black (0).

img_raw = plt.imread('data/cat_grass.jpg') / 255.0  # load the raw image and scale it to [0,1]
img_truth = np.round(plt.imread('data/truth.png'))  # load the ground truth image and round them to 0 or 1
train_cat = np.loadtxt('data/train_cat.txt', delimiter=',')
train_grass = np.loadtxt('data/train_grass.txt', delimiter=',')

Part 3.2 - Compute mean vector and covariance matrix

Both mean and covariance can be computed easily utilizing the numpy library. The command np.mean can take one additional parameter to indicate the axis. Axis 1 indicates we are looking for an average column instead of an average row.

def compute_mean_cov(train_data):
    # given the training data, compute the mean and covariance
    mu = np.mean(train_data, axis=1)
    sigma = np.cov(train_data)
    return len(train_data[0]), mu, sigma

K_cat, mu_cat, sigma_cat = compute_mean_cov(train_cat)
K_grass, mu_grass, sigma_grass = compute_mean_cov(train_grass)

Part 3.3 - Classification and decision making

This part builds the actual classifier and will be broken down into several parts: optimizations can be performed to reduce the runtime and three different implementations.

Part 3.3.1 - Obervations and optimizations

This part will be the most important part of the project. The following is a general procedure for classification:

  1. initialize the array to store the outputs
  2. precompute few constants to reduce the computation cost
  3. for each 8x8 patch
    1. flatten the patch to 64x1
    2. calculate f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{cat}\right)f\left(\text{cat}\right) and f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{grass}\right)f\left(\text{grass}\right) and compare
    3. assign the value to the output array: 1 means cat and 0 means grass
  4. return the output array

There are a few optimization we can do here.

  1. Comparing f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{cat}\right)f\left(\text{cat}\right) and f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{grass}\right)f\left(\text{grass}\right) holds the same result as comparing \log\left(f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{cat}\right)f\left(\text{cat}\right)\right) and \log\left(f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{grass}\right)f\left(\text{grass}\right)\right) since \log is a monotonically increasing function.
  2. \log\left(f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{class i}\right)f\left(\text{class i}\right)\right)=\log f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{class i}\right)+\log f\left(\text{class i}\right).
  3. \log f\left(\text{cat}\right) and \log f\left(\text{grass}\right) are constants throughout the compuation, so we can precompute both values once and simply use them after.
  4. \log\left(f_{\pmb{Z}\vert\text{class}}\left(\pmb{z}\vert\text{class i}\right)\right)=-\log\left((2\pi)^{d/2}\left\vert\pmb{\Sigma}_{\text{class i}}\right\vert^{1/2}\right)-\frac{1}{2}\left(\pmb{z}-\pmb{\mu}_{\text{class i}}\right)^T\pmb{\Sigma}_{\text{class i}}^{-1}\left(\pmb{z}-\pmb{\mu}_{\text{class i}}\right) holds true since \log(ab)=\log a+\log b and \log(\exp(x))=x.
  5. The entire first term in previous equation -\log\left((2\pi)^{d/2}\left\vert\pmb{\Sigma}_{\text{class i}}\right\vert^{1/2}\right) is an constant; therefore can be precomputed for once for the cat class and once for the grass class to avoid expensive calculation inside the loop.
  6. \pmb{\Sigma}_{\text{class i}}^{-1} is also a constant and can be precomputed before the loop for both classes.

Part 3.3.2 - Non-overlapping classifier

Since we are classifying each 8x8 patch, there are different ways of implementing the classifier. One of them will analyze all non-overlapping patches and classify all 64 pixels into one class for each patch (ignore the edges when the side is not divisible by 8). The shape of the input image for our example is (375, 500) in terms of the number of rows and number of columns, and the shape of the output classification matrix will be (368, 400) by ignoring the sides. The following code shows the implementation:

def classifier_nonoverlap(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, pi_cat, pi_grass):
    # find the dimension of the image
    M = len(img_raw)
    N = len(img_raw[0])

    # calculate log(f(class)) as described in item 3
    fcat = np.log(K_cat / (K_cat + K_grass))
    fgrass = np.log(K_grass / (K_cat + K_grass))

    # calculate the constent as described in item 4
    coef_cat = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_cat))))
    coef_grass = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_grass))))

    # calculate the inverse of the covariance matrix as described in item 6
    inv_cat = np.linalg.pinv(sigma_cat)
    inv_grass = np.linalg.pinv(sigma_grass)

    result = np.zeros((M // 8 * 8, N // 8 * 8))   # initialize the classification result matrix
    for i in range(M // 8):
        for j in range(N // 8):
            # extract the 8x8 patch, flatten it and find the difference to the mu of cat and grass
            z = img_raw[i * 8 : i * 8 + 8, j * 8 : j * 8 + 8]
            z = z.flatten('F')
            diff_cat = z - mu_cat
            diff_grass = z - mu_grass

            # calculate f_z|class(z|cat) and f_z|class(z|grass)
            fcat_z = fcat + coef_cat - 0.5 * np.matmul(np.matmul(np.transpose(diff_cat), inv_cat), diff_cat)
            fgrass_z = fgrass + coef_grass - 0.5 * np.matmul(np.matmul(np.transpose(diff_grass), inv_grass), diff_grass)

            # find out which class has higher probability and assign to all 64 pixels
            if (fcat_z > fgrass_z):
                result[i * 8 : i * 8 + 8, j * 8 : j * 8 + 8] = 1

    return result

Part 3.3.3 - Overlapping classifier

Another way is to classify each pixel based on one 8x8 patch and patches can overlap with one another. The resulting matrix’s size will be (368, 493). The following code shows the implementation:

def classifier_overlap(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass):
    # find the dimension of the image
    M = len(img_raw)
    N = len(img_raw[0])

    # calculate log(f(class)) as described in item 3
    fcat = np.log(K_cat / (K_cat + K_grass))
    fgrass = np.log(K_grass / (K_cat + K_grass))

    # calculate the constent as described in item 4
    coef_cat = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_cat))))
    coef_grass = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_grass))))

    # calculate the inverse of the covariance matrix as described in item 6
    inv_cat = np.linalg.pinv(sigma_cat)
    inv_grass = np.linalg.pinv(sigma_grass)

    result = np.zeros((M - 8 + 1, N - 8 + 1))   # initialize the classification result matrix

    for i in range(M - 8 + 1):
        for j in range(N - 8 + 1):
            # extract the 8x8 patch, flatten it and find the difference to the mu of cat and grass
            z = img_raw[i : i + 8, j : j + 8]
            z = z.flatten('F')
            diff_cat = z - mu_cat
            diff_grass = z - mu_grass

            # calculate f_z|class(z|cat) and f_z|class(z|grass)
            fcat_z = fcat + coef_cat - 0.5 * np.matmul(np.matmul(np.transpose(diff_cat), inv_cat), diff_cat)
            fgrass_z = fgrass + coef_grass - 0.5 * np.matmul(np.matmul(np.transpose(diff_grass), inv_grass), diff_grass)

            # find out which class has higher probability
            if (fcat_z > fgrass_z):
                result[i][j] = 1

    return result

Part 3.3.4 - Improved overlapping classifier

This is combining both methods for classification. The classifier will have overlapping 8x8 patches when looping through the input image but it will add the result of the patch to all 64 pixels. Therefore, most of the pixels will be classified 64 times with different patches while pixels on the edge will be classified less than that. In the end, each pixel will have a majority vote on which class it belongs to based on all classifications performed. The resulting image will have the same shape as the input image (375, 500).

def classify_improved_overlap(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass):
    # find the dimension of the image
    M = len(img_raw)
    N = len(img_raw[0])

    # calculate log(f(class)) as described in item 3
    fcat = np.log(K_cat / (K_cat + K_grass))
    fgrass = np.log(K_grass / (K_cat + K_grass))

    # calculate the constent as described in item 4
    coef_cat = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_cat))))
    coef_grass = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_grass))))

    # calculate the inverse of the covariance matrix as described in item 6
    inv_cat = np.linalg.pinv(sigma_cat)
    inv_grass = np.linalg.pinv(sigma_grass)

    result = np.zeros((M, N))   # initialize the classification result matrix
    count = np.zeros((M, N))

    for i in range(M - 8 + 1):
        for j in range(N - 8 + 1):
            # extract the 8x8 patch, flatten it and find the difference to the mu of cat and grass
            z = img_raw[i : i + 8, j : j + 8]
            z = z.flatten('F')
            diff_cat = z - mu_cat
            diff_grass = z - mu_grass

            # calculate f_z|class(z|cat) and f_z|class(z|grass)
            fcat_z = fcat + coef_cat - 0.5 * np.matmul(np.matmul(np.transpose(diff_cat), inv_cat), diff_cat)
            fgrass_z = fgrass + coef_grass - 0.5 * np.matmul(np.matmul(np.transpose(diff_grass), inv_grass), diff_grass)

            # find out which class has higher probability
            count[i : i + 8, j : j + 8] += 1
            if (fcat_z > fgrass_z):
                result[i : i + 8, j : j + 8] += 1

    return np.round(result / count)

Part 4 - Evaluating the Classifier

Given the classifier, we can evaluate the runtime and accuracy of the classifier. The following piece fo code is used for evaluation. The metric used here for accuracy is simply the number of correctly classified pixels to the total number of pixels. When evaluating the non-overlapping implementation, we are ignoring the right and bottom edges of the picture. When evaluating the overlapping implementation, we are ignoring the first 4 rows and columns, and the last 3 rows and columns; this is chosen to incorporate the idea that we are classifying each center pixel by the 8x8 patch around it. Switching the 3 and 4 around or using a different way of comparing the result with the ground truth is fine.

def eval(classifier, img_raw, img_truth, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, save_img=''):
    '''
    This method evaluates the classifier and calculate the error with the training model. The last argument is optional to save the result image.
    The first argument classifier is the classification function
    '''
    start_time = time.time()
    result = classifier(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass)
    end_time = time.time()
    accuracy = np.sum(np.sum(result == img_truth) / (len(result) * len(result[0])))
    
    if save_img != '':      # if the save file is given
        plt.imsave(save_img, result * 255, cmap='gray')

    return end_time - start_time, accuracy

M = len(img_raw)
N = len(img_raw[0])

runtime, accuracy = eval(classifier_nonoverlap, img_raw, img_truth[:M // 8 * 8, :N // 8 * 8], mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, 'non-overlapping.png')
print(f'Non-overalapping classifier takes {runtime:.3f} seconds to run and the accuracy is {accuracy:.3f}')

runtime, accuracy = eval(classifier_overlap, img_raw, img_truth[4:-3, 4:-3], mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, 'overlapping.png')
print(f'Overalapping classifier takes {runtime:.3f} seconds to run and the accuracy is {accuracy:.3f}')

runtime, accuracy = eval(classify_improved_overlap, img_raw, img_truth, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, 'overlapping_improved.png')
print(f'Improved overalapping classifier takes {runtime:.3f} seconds to run and the accuracy is {accuracy:.3f}')

Following are my outputs:

Non-overalapping classifier takes 0.054 seconds to run and the accuracy is 0.906
Overalapping classifier takes 1.400 seconds to run and the accuracy is 0.913
Improved overalapping classifier takes 2.192 seconds to run and the accuracy is 0.923

and those are my images:

Fig.1 - Output image from the non-overlapping classifier.


Fig.2 - Output image from the overlapping classifier.


Fig.3 - Output image from the improved overlapping classifier.


Part 5 - Discussion

The classifier performs well on the test image, but if you try with other images, it probably fails as shown in Figure 4. Why?

Fig.4 - Running the classifier on the image shown on the left and the result is shown on the right.


Couple possible reasons are:

  1. The training data are generated based on cat_grass.jpg this single image. Using such a biased training set and do well on the training data does not guarantee good performance on other images.
  2. The entire classifier is built on the assumption that the distribution of the texture of cat and grass are gaussian. In reality, this is unlikely, especially for things like a cat where there are so many different fur patterns.

Appendix - Source Code

import numpy as np
import matplotlib.pyplot as plt
import time

def compute_mean_cov(train_data):
    # given the training data, compute the mean and covariance
    mu = np.mean(train_data, axis=1)
    sigma = np.cov(train_data)
    return len(train_data[0]), mu, sigma

def classifier_nonoverlap(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, pi_cat, pi_grass):
    # find the dimension of the image
    M = len(img_raw)
    N = len(img_raw[0])

    # calculate log(f(class)) as described in item 3
    fcat = np.log(K_cat / (K_cat + K_grass))
    fgrass = np.log(K_grass / (K_cat + K_grass))

    # calculate the constent as described in item 4
    coef_cat = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_cat))))
    coef_grass = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_grass))))

    # calculate the inverse of the covariance matrix as described in item 6
    inv_cat = np.linalg.pinv(sigma_cat)
    inv_grass = np.linalg.pinv(sigma_grass)

    result = np.zeros((M // 8 * 8, N // 8 * 8))   # initialize the classification result matrix
    for i in range(M // 8):
        for j in range(N // 8):
            # extract the 8x8 patch, flatten it and find the difference to the mu of cat and grass
            z = img_raw[i * 8 : i * 8 + 8, j * 8 : j * 8 + 8]
            z = z.flatten('F')
            diff_cat = z - mu_cat
            diff_grass = z - mu_grass

            # calculate f_z|class(z|cat) and f_z|class(z|grass)
            fcat_z = fcat + coef_cat - 0.5 * np.matmul(np.matmul(np.transpose(diff_cat), inv_cat), diff_cat)
            fgrass_z = fgrass + coef_grass - 0.5 * np.matmul(np.matmul(np.transpose(diff_grass), inv_grass), diff_grass)

            # find out which class has higher probability and assign to all 64 pixels
            if (fcat_z > fgrass_z):
                result[i * 8 : i * 8 + 8, j * 8 : j * 8 + 8] = 1

    return result

def classifier_overlap(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass):
    # find the dimension of the image
    M = len(img_raw)
    N = len(img_raw[0])

    # calculate log(f(class)) as described in item 3
    fcat = np.log(K_cat / (K_cat + K_grass))
    fgrass = np.log(K_grass / (K_cat + K_grass))

    # calculate the constent as described in item 4
    coef_cat = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_cat))))
    coef_grass = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_grass))))

    # calculate the inverse of the covariance matrix as described in item 6
    inv_cat = np.linalg.pinv(sigma_cat)
    inv_grass = np.linalg.pinv(sigma_grass)

    result = np.zeros((M - 8 + 1, N - 8 + 1))   # initialize the classification result matrix

    for i in range(M - 8 + 1):
        for j in range(N - 8 + 1):
            # extract the 8x8 patch, flatten it and find the difference to the mu of cat and grass
            z = img_raw[i : i + 8, j : j + 8]
            z = z.flatten('F')
            diff_cat = z - mu_cat
            diff_grass = z - mu_grass

            # calculate f_z|class(z|cat) and f_z|class(z|grass)
            fcat_z = fcat + coef_cat - 0.5 * np.matmul(np.matmul(np.transpose(diff_cat), inv_cat), diff_cat)
            fgrass_z = fgrass + coef_grass - 0.5 * np.matmul(np.matmul(np.transpose(diff_grass), inv_grass), diff_grass)

            # find out which class has higher probability
            if (fcat_z > fgrass_z):
                result[i][j] = 1
    return result

def classify_improved_overlap(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass):
    # find the dimension of the image
    M = len(img_raw)
    N = len(img_raw[0])

    # calculate log(f(class)) as described in item 3
    fcat = np.log(K_cat / (K_cat + K_grass))
    fgrass = np.log(K_grass / (K_cat + K_grass))

    # calculate the constent as described in item 4
    coef_cat = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_cat))))
    coef_grass = -np.log(((2 * np.pi) ** (64 / 2) * np.sqrt(np.linalg.det(sigma_grass))))

    # calculate the inverse of the covariance matrix as described in item 6
    inv_cat = np.linalg.pinv(sigma_cat)
    inv_grass = np.linalg.pinv(sigma_grass)

    result = np.zeros((M, N))   # initialize the classification result matrix
    count = np.zeros((M, N))

    for i in range(M - 8 + 1):
        for j in range(N - 8 + 1):
            # extract the 8x8 patch, flatten it and find the difference to the mu of cat and grass
            z = img_raw[i : i + 8, j : j + 8]
            z = z.flatten('F')
            diff_cat = z - mu_cat
            diff_grass = z - mu_grass

            # calculate f_z|class(z|cat) and f_z|class(z|grass)
            fcat_z = fcat + coef_cat - 0.5 * np.matmul(np.matmul(np.transpose(diff_cat), inv_cat), diff_cat)
            fgrass_z = fgrass + coef_grass - 0.5 * np.matmul(np.matmul(np.transpose(diff_grass), inv_grass), diff_grass)

            # find out which class has higher probability
            count[i : i + 8, j : j + 8] += 1
            if (fcat_z > fgrass_z):
                result[i : i + 8, j : j + 8] += 1

    return np.round(result / count)

def eval(classifier, img_raw, img_truth, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, save_img=''):
    '''
    This method evaluates the classifier and calculate the error with the training model. The last argument is optional to save the result image.
    The first argument classifier is the classification function
    '''
    start_time = time.time()
    result = classifier(img_raw, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass)
    end_time = time.time()
    accuracy = np.sum(np.sum(result == img_truth) / (len(result) * len(result[0])))
    
    if save_img != '':      # if the save file is given
        plt.imsave(save_img, result * 255, cmap='gray')

    return end_time - start_time, accuracy


if __name__ == '__main__':
    img_raw = plt.imread('cat_grass.jpg') / 255.0  # load the raw image and scale it to [0,1]
    img_truth = np.round(plt.imread('truth.png'))  # load the ground truth image and round them to 0 or 1
    train_cat = np.loadtxt('train_cat.txt', delimiter=',')
    train_grass = np.loadtxt('train_grass.txt', delimiter=',')

    K_cat, mu_cat, sigma_cat = compute_mean_cov(train_cat)
    K_grass, mu_grass, sigma_grass = compute_mean_cov(train_grass)

    M = len(img_raw)
    N = len(img_raw[0])

    runtime, accuracy = eval(classifier_nonoverlap, img_raw, img_truth[:M // 8 * 8, :N // 8 * 8], mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, 'non-overlapping.png')
    print(f'Non-overalapping classifier takes {runtime:.3f} seconds to run and the accuracy is {accuracy:.3f}')

    runtime, accuracy = eval(classifier_overlap, img_raw, img_truth[4:-3, 4:-3], mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, 'overlapping.png')
    print(f'Overalapping classifier takes {runtime:.3f} seconds to run and the accuracy is {accuracy:.3f}')

    runtime, accuracy = eval(classify_improved_overlap, img_raw, img_truth, mu_cat, mu_grass, sigma_cat, sigma_grass, K_cat, K_grass, 'overlapping_improved.png')
    print(f'Improved overalapping classifier takes {runtime:.3f} seconds to run and the accuracy is {accuracy:.3f}')

© 2020. All rights reserved.