Description
Note: You can use any 3rd party libraries and built-in functions
Question 1: find, read and summarize a paper (40 pts)
Find a paper (related to Perception/Vision) that you are interested in from:
https://openaccess.thecvf.com/CVPR2021?day=all https://openaccess.thecvf.com/ICCV2021?day=all https://paperswithcode.com/
Requirement: up to 1 page with below info:
(You should never literately copy any sentences from any digital sources for your report)
- The link of this paper.
- In the structure of this paper, where can you find below info?
- Background
- Motivations
- Short Summary of the proposed work
- Contribution highlights
- Problem formulation
- Conclusion
- What problem it is resolving?
- What are the input and output of the proposed method?
- The challenges of this problem?
- What are the purposes for the first and second figures?
- What new concept its method introduced?
- What counterparts it compared to and links of typical counterparts?
- What aspect are included in the Experiment sections?
- What results it concludes?
Option 1: Python
Question 2: train an instance segmentation model with PyTorch tutorial and Google Colab; or you can deploy it in Palmetto Cluster. (60 pts)
- Show screenshots of successful setup, training, and inference on the colab. (15 points)
- Inference different images in the test set and show screenshots (5 points)
- Inference on your own image (10 points)
- Plot the segmentation metric AP@[IoU=0.50] against the number of training Epochs on a Graph. (15 points)
(TIPS: if you cannot save accuracy during training, you can manually collect it from outputs.)
- Change the batch size, optimizer, learning rate etc… Plot and analyze its influence on accuracy. (15 points)
- Could you improve the network model, train it for better accuracy? (Optional, 5 points) (This question is optional. Extra 5 points until reach the cap of 100)
Option 2: Matlab
Your Clemson credential has fully license to Matlab, for computing platform: – You can use ‘MATLAB Online’ https://matlab.mathworks.com/ – Or you use Matlab in your local computer.
Reference:
Deep Learning Toolbox
https://www.mathworks.com/help/deeplearning/index.html Deep Learning Onramp
https://matlabacademy.mathworks.com/details/deep–learning–onramp/deeplearning
Question 2: ‘on pretrained models’ (20 pts)
- Select a few (like three) of Pretrained models; Explain what is the input/output, and which paper (if any) it is introduced? Use deepNetworkDesigner to visualize the models.
Reference:
Pretrained Deep Neural Networks
https://www.mathworks.com/help/deeplearning/ug/pretrained–convolutional–neuralnetworks.html
- For the selected pretrained models; Test it using some of the images that you download from the web; Reference:
Classify Webcam Images Using Deep Learning https://www.mathworks.com/help/deeplearning/ug/classify–images–from–webcam–using–deeplearning.html
Show and discuss your accuracy: like you choose same type of objects with different background, and see their accuracy difference, etc.
Question 3: ‘further training a pretrained models’ (40 pts)
- Apply transfer learning to retrain a model to classify a new set of images.
Demo this example, reference:
Train Deep Learning Network to Classify New Images
https://www.mathworks.com/help/deeplearning/ug/train–deep–learning–network–to–classify–newimages.html
Explain its input and output; Explain what are ‘Batch’, ‘Epoch’, ‘Iteration’?
Try a few different ‘miniBatchSize’, and discuss its accuracy difference.
- For the demo of a selected parameters, select a partial of the testing/validation dataset statistically summarize its results for:
Explain what are:
True positive (TP)
True negative (TN)
False positive (FP)
False negative (FN)
Provide its result of TP, TN, FP, FN and Accuracy, Precision, Recall, F1-score
