Question 1: Basic Questions
(a) [2 pts] For a binary classification task implemented via a neural network, we usually use a sigmoid
activation function in the output unit. What is the activation function used in the ou
...
Question 1: Basic Questions
(a) [2 pts] For a binary classification task implemented via a neural network, we usually use a sigmoid
activation function in the output unit. What is the activation function used in the output unit for a
multi-class classification task? Write the formula.
Softmax activation function.
σ(zi) = ezi
PK j=1 ezj for i = 1, 2, . . . , K (1)
(b) [2 pts] Suppose there is a model that has a large loss evaluated on both train data and test data after
finishing the whole training process, what should we do? (list two methods).
Increase the complexity of the model and the number of epochs.
(c) [4 pts] Suppose there are 120 training examples for 12-Fold Cross-Validation error. We need to calculate
error N1 times, build a model with data of size N2, and test the model on the data of size N3. What
are the appropriate numbers for N1, N2, and N3?
12, 110, 10
(d) [4 pts] Bayes Rule: There is a company for selling 4000 cups, 8000 notebooks, and 12000 pencils. The
return probabilities of a cup, notebook, and pencil are 0.01, 0.03, and 0.05, respectively. If there is one
of the returned items, find the probability that it is a notebook.
Let E1 = event of a returned item being a cup,
E2 = event of a returned item being a notebook,
E3 = event of a returned item being a pencil,
A = event of the returned item.
P(E1) = 4000/(4000 + 8000 + 12000) = 1 6.
P(E2) = 8000/(4000 + 8000 + 12000) = 1 3.
P(E3) = 12000/(4000 + 8000 + 12000) = 1 2.
P(A|E1) = 0.01, P(A|E2) = 0.03, and P(A|E3) = 0.05,
Therefore, P(E1|A) = P (A|E1)P (E1)
P (A|E1)P (E1)+P (A|E2)P (E2)+P (A|E3)P (E3) =
3
11
(e) [4 pts] In which one of the following figures do you think the hypothesis has overfit and underfit the
training set? List at least TWO methods to mitigate the problems of overfitting and underfitting,
respectively.
[Show More]
