Description
Human pose recognition, and related tasks like gesture recognition and action recognition, are important tasks for AI systems that interact with people. There are many algorithms for classifying pose; for example, deep convolutional neural networks can be trained to classify pose directly from images. However, these methods tend to be “black boxes” and it’s difficult to understand what features they are using. Another approach, which we will use in this Project, uses neural networks to first identify keypoints corresponding to the main parts of the body (as shown above), and then learns to recognize pose based on the positions of these body parts.
In this project, you will implement a supervised na¨ıve Bayes learner to classify pose from keypoints provided by a deep convolutional neural network. You will train, test, and evaluate your classifier on a provided dataset, and then you will have a choice of either extending this basic model in various ways, or using it to answer some conceptual questions about na¨ıve Bayes.
Naive Bayes classifier
There are some suggestions for implementing your learner in the “Na¨ıve Bayes” and “Discrete & Continuous Data” lectures, but ultimately, the specifics of your implementation are up to you. Your implementation must be able to perform the following functions:
- preprocess() the data by reading it from a file and converting it into a useful format for training and testing
- train() by calculating prior probabilities and likelihoods from the training data and using these to build a naive Bayes model
- predict() classes for new items in a test dataset
- evaluate() the prediction performance by comparing your model’s class outputs to ground truth labels
Your implementation should be able to handle numeric attributes and it should assume that numeric attributes are Gaussian-distributed. Your model will not be expected to handle nominal attributes. Your implementation should actually compute the priors, likelihoods, and posterior probabilities for the na¨ıve Bayes model and not simply call an existing implementation such as GaussianNB from scikit-learn.
Data
The data for this assignment is drawn from a yoga pose classification dataset created by Anastasia Marchenkova and released online here. Separate training and test datasets are provided; please report your results on the provided test set.
The pose keypoints were produced by a computer vision algorithm based on OpenPose. The algorithm identifies 11 keypoints on the body and returns their x and y values. The data is provided as a csv file; the first column is the name of the yoga pose and the remaining columns are the keypoints (11 x values followed by 11 y values). The body parts represented by each of the 11 keypoints are shown to the right.
Since the keypoint data was generated by a computer vision algorithm, it may contain some errors. Some instances have missing keypoints because a part of the body was not visible in the original image or the algorithm failed to detect it. Missing keypoints have x and y values of 9999.
Keypoint diagram
Implementation tips
In the training phase of your algorithm, you will need to set up data structures to hold the prior probabilities for each class, and the likelihoods P(xi|cj) for each attribute xi in each class cj. Recall that you will need two parameters (mean and standard deviation) to define the Gaussian distribution for each attribute × class. A 2D array may be a convenient data structure to store these parameters.
Multiplying many probabilities in the range (0,1] can result in very low values and lead to underflow (numbers smaller than the computer can represent). When implementing a na¨ıve Bayes model, it is strongly recommended to take the log() of each probability and sum them instead of multiplying. E.g., instead of computing:
P(cj)YP(xi|cj) (1)
i
compute:
log(P(cj)) + Xlog(P(xi|cj)) (2)
i
Questions
The following problems are designed to pique your curiosity when running your classifier(s) over the given dataset and suggest methods for improving or extending the basic model.
- Since this is a multiclass classification problem, there are multiple ways to compute precision,recall, and F-score for this classifier. Implement at least two of the methods from the “Model Evaluation” lecture and discuss any differences between them. (The implementation should be your own and should not just call a pre-existing function.)
- The Gaussian na¨ıve Bayes classifier assumes that numeric attributes come from a Gaussian distribution. Is this assumption always true for the numeric attributes in this dataset? Identify some cases where the Gaussian assumption is violated and describe any evidence (or lack thereof) that this has some effect on the NB classifier’s predictions.
- Implement a kernel density estimate (KDE) na¨ıve Bayes classifier and compare its performance to the Gaussian na¨ıve Bayes classifier. Recall that KDE has kernel bandwidth as a free parameter – you can choose an arbitrary value for this, but a value in the range 5-25 is recommended. Discuss any differences you observe between the Gaussian and KDE na¨ıve Bayes classifiers. (As with the Gaussian na¨ıve Bayes, this KDE na¨ıve Bayes implementation should be your own and should not just call a pre-existing function.)
- Instead of using an arbitrary kernel bandwidth for the KDE na¨ıve Bayes classifier, use random hold-out or cross-validation to choose the kernel bandwidth. Discuss how this changes the model performance compared to using an arbitrary kernel bandwidth.
- Na¨ıve Bayes ignores missing values, but in pose recognition tasks the missing values can be informative. Missing values indicate that some part of the body was obscured and sometimes this is relevant to the pose (e.g., holding one hand behind the back). Are missing values useful for this task? Implement a method that incorporates information about missing values and demonstrate whether it changes the classification results.
- Engineer your own pose features from the provided keypoints. Instead of using the (x,y) positions of keypoints, you might consider the angles of the limbs or body, or the distances between pairs of keypoints. How does a na¨ıve Bayes classifier based on your engineered features compare to the classifier using (x,y) values? Please note that we are interested in explainable features for pose recognition, so simply putting the (x,y) values in a neural network or similar to get an arbitrary embedding will not receive full credit for this question. You should be able to explain the rationale behind your proposed features. Also, don’t forget the conditional independence assumption of na¨ıve Bayes when proposing new features – a large set of highly-correlated features may not work well.
If you are in a group of 1, you will respond to two questions of your choosing (two responses in total). If you are in a group of 2, you will respond to four questions of your choosing (four responses in total). A response to a question should take about 150–250 words, and make reference to the data wherever possible. We strongly recommend including figures or tables to support your responses.
