CS 234 Midterm - Winter 2018-19
**Do not turn this page until you are instructed to do so.
Instructions
Please answer the following questions to the best of your ability. Read all the questions first before
answering
...
CS 234 Midterm - Winter 2018-19
**Do not turn this page until you are instructed to do so.
Instructions
Please answer the following questions to the best of your ability. Read all the questions first before
answering. You have 80 minutes to complete the exam. The exam is closed-book and closed-internet.
Additionally use of all electronic items during the exam is prohibited. However, you may use one
one-sided letter-sized page of notes as reference. All of the intended answers can be written well within
the space provided. Good luck!
Stanford University Honor Code
The following is a statement of the Stanford University Honor Code:
1. The Honor Code is an undertaking of the students, individually and collectively: (1) that they
will not give or receive aid in examinations; that they will not give or receive unpermitted aid in
class work, in the preparation of reports, or in any other work that is to be used by the instructor
as the basis of grading; (2) that they will do their share and take an active part in seeing to it
that others as well as themselves uphold the spirit and letter of the Honor Code.
2. The faculty on its part manifests its confidence in the honor of its students by refraining from
proctoring examinations and from taking unusual and unreasonable precautions to prevent the
forms of dishonesty mentioned above. The faculty will also avoid, as far as practicable, academic
procedures that create temptations to violate the Honor Code.
3. While the faculty alone has the right and obligation to set academic requirements, the students
and faculty will work together to establish optimal conditions for honorable academic work.
By signing your name below, you acknowledge that you have abided by the Stanford Honor Code while
taking this exam.
Signature:
Name:
SUNet ID:
Grading (For Midterm Graders Only)
Question # 1 2 3 4 Total
Maximum Points 25 10 20 25 80
Student Grade
1Question 1 – True/False and Short Answers [25 pts]
(A) Circle True or False. Provide a one-sentence justification for each question. [2 pts each]
1. TRUE False An optimal policy of an infinite-horizon MDP is always stationary (i.e. it does
not vary with the time step).
Solution: True. Infinite-horizon MDPs are guaranteed to have a stationary optimal policy.
2. True FALSE When using a linear value function approximation, both Monte Carlo and
TD(0) policy evaluations achieve the minimum mean squared error possible
at convergence.
Solution: False. They may converge to different value functions unless the environment is
Markovian and both of them achieve zero mean squared error (which is not necessarily true
due to error from function approximation).
3. TRUE False All discounted MDPs with finite states and action spaces with bounded reward
(i.e. jR(s; a)j ≤ Rmax for some Rmax ≥ 0) have bounded returns.
Solution: True. Since the reward is bounded and γ < 1, return is a bounded geometric
series.
4. TRUE False Monte Carlo policy evaluation does not require the environment to be Markovian.
[Show More]
