Description
Perform linear regression of a noisy sinewave using a set of gaussian basis
005 functions with learned location and scale parameters. Model parameters are
006 learned with stochastic gradient descent. Use of automatic differentiation is
007
008 required. Hint: note your limits!
009
010
011 Problem Statement Consider a set of scalars {x1,x2,…,xN} drawn from U(0,1) 012 and a corresponding set {y1,y2,…,yN} where:
013
| 014
015 016 017 018 019 020 021 022 |
yi = sin(2πxi)+ ϵi
and ϵi is drawn from N(0,σnoise). Given the following functional form: yˆi = ∑wjϕj (xi | µj,σj)+ b M j=1 with: |
(1)
(2) |
023
024(3)
025
| 026
027 |
find estimates ˆb, {µˆj}, {σˆj}, and {wˆj} that minimize the loss function: |
028
029(4)
030
031 for all (xi,yi) pairs. Estimates for the parameters must be found using stochastic
032 gradient descent. A framework that supports automatic differentiation must be
033
034 used. Set N = 50,σnoise = 0.1. Select M as appropriate. Produce two plots. First,
035 show the data-points, a noiseless sinewave, and the manifold produced by the
036
regression model. Second, show each of the M basis functions. Plots must be of
037
038 suitable visual quality.
