ST308
Bayesian Inference
Suitable for all candidates
Instructions to candidates
This paper contains four questions. Answer ALL FOUR. All questions will be given equal
weight (25%).
The marks in brackets reflect mar
...
ST308
Bayesian Inference
Suitable for all candidates
Instructions to candidates
This paper contains four questions. Answer ALL FOUR. All questions will be given equal
weight (25%).
The marks in brackets reflect marks for each question.
Time allowed Reading Time: None
Writing Time: 2 hours
You are supplied with: A table of common distributions in the back of this exam paper
You may also use: No additional material
Calculators: Calculators are allowed in this exam
LSE LT2022/ST308 Page 1 of 111. (a) Briefly comment on how the Frequentist risk is different than the Posterior
risk and the Bayes risk. Also describe how one can find a Bayes estimator.
Use no more than five sentences. [6 marks]
Answer: [Direct coursework]
(b) Let x be an observation with probability density function f(xjθ), where θ is
a one-dimensional parameter. Also, denote with π(θjx) and π(θ) the posterior and prior distributions of θ respectively. We are interested in finding an
estimator for θ, denoted by a according to the following loss function
L(a; θ) = (a − θ)2 + λ(a − c)2;
where λ(> 0) and c are two known constants.
i. Find the Bayes estimator for the above loss function. Assume that you
are able to interchange between integration and differentiation.
[13 marks]
Answer:
The Bayes estimator minimises the posterior risk which is defined as
ρa; π(θ) = EθjxL(a; θ) = Z (a − θ)2 + λ(a − c)2π(θjx)dθ:
In order to find the function a that minimises ρa; π(θ) we look at the
system
@
@aρa; π(θ) = 0 ! @a @ Z (a − θ)2 + λ(a − c)2π(θjx)dθ = 0 !
Z @a @ (a − θ)2 + λ(a − c)2π(θjx)dθ = 0 !
2 Z (a − θ + λa − λc)π(θjx)dθ = 0 !
a(1 + λ) − λc Z π(θjx)dθ − Z θπ(θjxdθ = 0 !
a(1 + λ) = E(θjx) + λc ! a = E(θjx) + λc
1 + λ
ii. Describe a real-world setting where the above loss function may be
useful. [6 marks]
Answer: [Students have not seen such questions]
Maybe c is a commonly used value and so a different value would be
costly to implement. So we might want to balance estimates near the
truth. As λ goes to 0 we are moving towards the sample mean which
comes from the squared error loss function.
LSE LT2022/ST308 Page 2 of 112. Consider a sample x = (x1; : : : ; xn) of independent random variables, identically
distributed from the Rayleigh(θ) distribution. The probability density function for
each xi is
f(xijθ) = θxi exp(−0:5θx2 i ); xi > 0;
where θ > 0 is a unknown parameter.
(a) Choose a suitable prior for θ and derive the corresponding posterior. Justify
your choice of prior. [5 marks]
Answer: The likelihood can be written as
L(θjx) = f(xjθ) =
nY i
=1
θxi exp(−0:5θx2 i ) / θn exp −0:5θ
nX i
=1
x2
i! ;
Consider a Gamma(α; β) prior for θ. We can write
π(θ) / θα−1 exp(−βθ):
Hence the corresponding posterior will be proportional to
π(θjx) / θα−1 exp(−βθ)θn exp −0:5θ
nX i
=1
x2
i!
= θn+α−1 exp (−θ β + 0:5 Xi=1 n x2 i!)
which can be recognised as the kernel of the Gamma(n + α, β + 0:5 Pn i=1 x2 i )
distribution.
In our case there is no prior information available so the prior parameters (α,
β) can be chosen such that the prior variance is high. For example choosing
α = β = 0:001 will correspond to a prior mean of 1 and prior variance of 103.
(b) Derive the Jeffreys prior for θ. Use it to obtain the corresponding posterior
distribution. [7 marks]
Answer: The log-likelihood can be written as
‘(θjx) = n log(θ) − 0:5θ
nX i
=1
x2
i :
In order to find Jeffreys prior we calculate
@‘(θjx)
@θ =
n θ
− 0:5
nX i
=1
x2
i ;
@2‘(θjx)
@θ2 = −
n 2θ
;
and the Fisher information
[Show More]
