[SOLVED] ML-Homework 4 Logistic regression and EM algorithm

Description

1. Logistic regression

Input:

1. (number of data points)
2. ( : mean, : variance)

Function:

1. Generate data point: , where  and  are independently sampled from  and
2. Generate data point: , where  and  are independently sampled from  and
3. Use Logistic regression to separate  and     . You should implement both Newton’s and steepest gradient descent method during optimization.

In other words, when the Hessian is singular, use steepest descent for instead. You should come up with a reasonable rule to determine convergence.(a simple run out of the loop should be used as the ultimatum)    Output:

1. The confusion matrix and the sensitivity and specificity of the logistic regression applied to the training data .
2. Visualization

Plot the ground truth

Plot the predict result

2. EM algorithm

Input: MNIST training data and label sets. (Same as HW02) Function:

1. Binning the gray level value into two bins. Treating all pixels as random variables following Bernoulli distributions. Note that each pixel follows a different Binomial distribution independent to others.

• Use EM algorithm to cluster each image into ten groups. You should come up with a reasonable rule to determine convergence. (a simple run out of the loop should be used as the ultimatum) Output:

1. For each digit, output a confusion matrix and the sensitivity and specificity of the clustering applied to the training data.
2. Print out the imagination of numbers in your classifier

Just like before, about the details please refer to HW02

Hint: The algorithm is a kind of unsupervised learning, so the labels are not used during training. But you can use these labels to help you to figure out which class belongs to which number.

In other words, you should find a way to assign label to each class which you classified before you compute the confusion matrix