Description
Q1 mark scheme (40 pts)
At least 2 algorithms should be tested. If only 1 is tested then the maximum points for the question is 20. You can obtain full marks using 2 algorithms plus the baseline.
(5pts) Overall presentation of the report, including use of appropriate sections, plots, diagrams, or tables to make your point. Do not include code snippets in the report. Instead, describe in words or equations what you are implementing. Format equations correctly.
(3pts) Picking a suitable type of algorithm (classification/regression/clustering) and justifying this choice. The lectures and worksheet from week 13 will be helpful here.
- pts) An appropriate choice of performance metric (e.g.: accuracy/precision/mean squared error etc) and justification. The lectures and worksheet from week 13 will be helpful here.
- pts) Discussion of the kind of baseline to compare against. (sklearn has a module dummy which may be useful in generating a baseline).
(15 pts) Use of an appropriate method to select the hyperparameters of the chosen algorithms. The explanation of which hyperparameters are selected should be backed up with e.g. tables and plots to show which hyperparameter values were chosen and why. Please choose at least one model that uses hyperparameters so that you can show your knowledge in this area. If you choose one model without hyperparameters then please explain in a couple of sentences what the benefits of choosing a model without hyperparameters are. The lectures and worksheet from week 13 will be helpful here.
Breakdown
- 3 pts: Show that you understand what hyperparameters are and how they can be selected.
- 5 pts: Look at the effects of different hyperparameter choices on the performance of your models.
- 5 pts: Present the effects of the different hyperparameter choices on the performance of your models using tables, plots, or other presentation.
- 2 pts: State what hyperparameter choices you make and why.
(10 pts) Training and testing the performance of the models in a way that shows whether the models are able to generalise to unseen data and that ensures that the performance of the models is robust. The lectures and worksheet from week 13 will be helpful here.
- 4 pts: Train models and select hyperparameters in a way that gives robust performance 3 pts: Test the performance of your models and compare their performance
- 3 pts: Make sure your models are tested in a way that shows whether they are able to generalise to unseen data
Recommended structure of the short report
The short report should be no more than 4 pages. Shorter is fine. You should use LATEX, MS Word, or a similar text editor to prepare the report and submit it as a pdf document. • Introduction: State what the problem is. State what kind of algorithm needs to be used (classification/regression/clustering) and explain why that kind of algorithm needs to be used.
- Methods: State which specific algorithms you will use. State which performance metric you will use and why. Describe the baseline that you will measure your algorithms against. Describe how you will choose the hyperparameters of the algorithms. Explain which hyperparameters you have selected for each model using tables or plots to illustrate your decision.
- Results: Report the results of your models. Use tables or plots as appropriate to illustrate your results.
Question 2: 10pts
The Flickr-Faces-HQ (FFHQ) dataset is available at https://github.com/NVlabs/ffhq-dataset and described in appendix A of the paper Karras et al. [2019]. NB: You do not need to read the whole paper. I have provided a template with a selection of the datasheet questions in sections 3.2 (Composition), 3.3 (Collection Process) and 3.5 (Uses) of the paper Gebru et al. [2021]. Please provide answers to the questions in the template.
Page guide: The template is 2 pages long. The completed template with your answers should be about 3 pages long – most questions need a sentence or two answer. Some may need longer or shorter answers.
References
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daum´e Iii, and Kate Crawford. Datasheets for datasets. Communications of the ACM, 64(12):86–92, 2021. URL https://arxiv.org/abs/1803.09010.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019. URL https://arxiv.org/abs/ 1812.04948.
