Description
In this assignment you will investigate a real dataset by implementing multiple classification algorithms. You will first pre-process the dataset by replacing missing values and normalising the dataset with a minmax scaler. You will then evaluate multiple classification algorithms: K-Nearest Neighbour, Logistic Regression, Naïve Bayes, Decision Tree, Support Vector Machine and ensembles – Bagging, Boosting (AdaBoost, Gradient Boosting) and Random Forest. These will then be evaluated using the stratified 10fold cross validation method. You will also apply a grid search to find the best parameters for some of the classifiers.
Programming language
This assignment must be written in Python. It should use the classification algorithms from the sklearn (scikit-learn) library used in the tutorials. You can also use the numpy and pandas libraries.
1. Dataset preparation
The data
The dataset for this assignment is the Breast Cancer Wisconsin. It contains 699 examples described by 9 numeric attributes. There are two classes – class1 and class2. The features are computed from a digitized image of a fine needle aspirate of a breast mass of the subject. Benign breast cancer tumours correspond to class1 and malignant breast cancer tumours correspond to class2.
The dataset should be downloaded from Canvas: breast-cancer-wisconsin.csv. This file includes the attribute (feature) headings and each row corresponds to one individual. Missing attributes in the dataset are recorded with a ‘?’.
Data pre-processing
You will need to pre-process the dataset, before you can apply the classification algorithms. Three types of pre-processing are required:
- The missing attribute values should be replaces with the mean value of the column using impute.SimpleImputer.
- Normalisation of each attribute should be performed using a min-max scaler to normalise the values between [0,1] with preprocessing.MinMaxScaler.
- The classes class1 and class2 should be changed to 0 and 1
- The value of each attribute should be formatted to 4 decimal places using .4f.
The correctness of your pre-processed data will be automatically tested by PASTA. Thus, you need to make sure that you follow the input and output instructions carefully.
2. Classification algorithms with 10-fold cross-validation
You will now apply multiple classifiers to the pre-processed dataset, in particular: Nearest Neighbor, Logistic Regression, Naïve Bayes, Decision Tree, Bagging, Ada Boost and Gradient Boosting. All classifiers should use the sklearn modules from the tutorials. All random states in the classifiers should be set to random_state=0.
You need to evaluate the performance of these classifiers using 10-fold cross validation from sklearn.model_selection.StratifiedKFold with these options: cvKFold=StratifiedKFold(n_splits=10, shuffle=True, random_state=0)
For each classifier, write a function that accepts the required input and that outputs the average crossvalidation score: def exampleClassifier(X, y, [options]):
…
return scores, scores.mean()
where X contains the attribute values and y contains the class (as in the tutorial exercises).
More specifically, the headers of the functions for the classifiers are given below:
K-Nearest Neighbour
def kNNClassifier(X, y, K)
It should use the KNeighboursClassifier from sklearn.neighbours.
Logistic Regression
def logregClassifier(X, y)
It should use LogisticRegression from sklearn.linear_model.
Naïve Bayes
def nbClassifier(X, y)
It should use GaussianNB from sklearn.naive_bayes
Decision Tree
def dtClassifier(X, y)
It should use DecisionTreeClassifier from sklearn.tree, with information gain (the entropy criterion)
Ensembles
def bagDTClassifier(X, y, n_estimators, max_samples, max_depth) def adaDTClassifier(X, y, n_estimators, learning_rate, max_depth) def gbClassifier(X, y, n_estimators, learning_rate)
These functions should implement Bagging, Ada Boost and Gradient Boosting using BaggingClassifier, AdaBoostClassifier and GradientBoostingClassifier from sklearn.ensemble. All should combine Decision Trees with information gain.
3. Parameter Tuning
For two other classifiers, Linear SVM and Random Forest, we would like to find the best parameters using grid search with 10-fold stratified cross validation (GridSearchCV in sklearn).
The split into training and test subsets should be done using train_test_split from sklearn.model_selection with stratification and random_state=0 (as in the tutorials but with random_state=0).
Hint: You need to pass StratifiedKFold as an argument to GridSearchCV, not cv=10 as in the tutorials. This ensures that random_state=0.
Write the following functions:
Linear SVM
def bestLinClassifier(X,y)
It should use SVC from sklearn.svm.
The grid search should consider the following values for the parameters C and gamma: C = {0.001, 0.01, 0.1, 1, 10, 100} gamma = {0.001, 0.01, 0.1, 1, 10, 100}
The function should print the best parameters found, the best-cross validation accuracy score and the best test set accuracy score, see Section 4.
Random Forest
def bestRFClassifier(X,y)
It should use RandomForestClassifier from sklearn.ensemble with information gain and max_features set to ‘sqrt’.
The grid search should consider the following values for the parameters n_estimators and max_leaf_nodes:
n_estimators = {10, 30} max_leaf_nodes = {4, 16}
The function should print the best parameters found, best-cross validation accuracy score and best test set accuracy score, see Section 4.
