Description
Perform linear regression of a noisy sinewave using a set of gaussian basis
functions with learned location and scale parameters. Model parameters are
learned with stochastic gradient descent. Use of automatic differentiation is
required. Hint: note your limits!
Problem Statement Consider a set of scalars {x1,x2,…,xN} drawn from U(0,1) 012 and a corresponding set {y1,y2,…,yN} where:
| yi = sin(2πxi)+ ϵi
and ϵi is drawn from N(0,σnoise). Given the following functional form: yˆi = ∑wjϕj (xi | µj,σj)+ b M j=1 with: |
(1)
(2) |
| 026
027 |
find estimates ˆb, {µˆj}, {σˆj}, and {wˆj} that minimize the loss function: |
for all (xi,yi) pairs. Estimates for the parameters must be found using stochastic
gradient descent. A framework that supports automatic differentiation must be
used. Set N = 50,σnoise = 0.1. Select M as appropriate. Produce two plots. First,
show the data-points, a noiseless sinewave, and the manifold produced by the
regression model. Second, show each of the M basis functions. Plots must be of
−4 −2 0 2 4 −4 −2 0 2 4 x x
Figure 1: Example plots for models with equally spaced sigmoid and gaussian basis functions.
