[SOLVED] AI6127 - Deep Neural Networks for Natural Language Processing - Assignment 2

Description

Question One [50 marks]

1. 9. Rerun the implementations of machine translation (MT) of Tutorial 6 with the multiple parallel datasets at http://www.statmt.org/wmt14/training-parallel-nctgz as follows: (25 marks)
      1. Refer to http://www.statmt.org/wmt14/translation-task.html#download

(News Commentary) for details ii. Randomly split a dataset into 5 subsets (S₁, …, S₅). Run training/testing for 5 times as follows: (5 marks)

1. Select S_i (i=1,…,5) as test dataset and the other 4 subsets as training dataset. Train a model with the training dataset and evaluate the model against the test dataset in terms of BLEU (BLEU-1, BLEU-2, BLEU-3).
2. Report the average of the 5 BLEU scores for each dataset.

iii. Rerun the implementations for Question 3 (10 marks) and Question 6 (10 marks) of Tutorial 6.

1. Do not use the filters of Q 1.b of Tutorial 6 (i.e. MAX_LENGTH, prefixes)
2. For Question 6, if you do not find parameters that outperform the MT implementation of Question 3 for a dataset, specify which parameters you have tested and discuss why they were not effective for the dataset.
3. Note that the file has 4 parallel datasets (CS-EN, DE-EN, FR-EN, RUEN). Rerun the implementations for all the 4 datasets, English as target language.

1. The Attention Decoder of Tutorial 6 is different from the attention decoder of the lecture (Sequence-to-sequence with attention). (25 marks)
   1. Explain the difference (5 marks)
   2. Modify the Attention Decoder of Tutorial 6 to the attention decoder of Lecture 6. Evaluate it for Question 3 of Tutorial 6 against the 4 parallel datasets aforementioned. Compare its evaluation results with the results from Question 2.a of this assignment, and discuss why. (10 marks)

• Replace the dot-product attention of Question 2.b.ii of this assignment to the following attention variants: (10 marks)
  1. Multiplicative attention:
     1. Where
  2. Additive attention:
     1. Where is a weight vector. d₃ (the attention dimensionality) is a hyper-parameter.
  3. Refer to https://ruder.io/deep-learning-nlp-bestpractices/index.html#attention for more details of the variants

1

Deep Neural Networks for Natural Language Processing (AI6127)

     Assignment 2: Sequence-to-sequence with attention

4. Compare evaluation results for Question 3 of Tutorial 6 with the results from Questions 2.a and 2.b.ii of this assignment, and discuss why.

2