when might overfitting occur Correct Answer-when the # of factors is close to or larger than the # of
data points causing the model to potentially fit too closely to random effects
Why are simple models better than com
...
when might overfitting occur Correct Answer-when the # of factors is close to or larger than the # of
data points causing the model to potentially fit too closely to random effects
Why are simple models better than complex ones Correct Answer-less data is required; less chance of
insignificant factors and easier to interpret
what is forward selection Correct Answer-we select the best new factor and see if it's good enough (R^2,
AIC, or p-value) add it to our model and fit the model with the current set of factors. Then at the end we
remove factors that are lower than a certain threshold
what is backward elimination Correct Answer-we start with all factors and find the worst on a supplied
threshold (p = 0.15). If it is worse we remove it and start the process over. We do that until we have the
number of factors that we want and then we move the factors lower than a second threshold (p = .05)
and fit the model with all set of factors
what is stepwise regression Correct Answer-it is a combination of forward selection and backward
elimination. We can either start with all factors or no factors and at each step we remove or add a
factor. As we go through the procedure after adding each new factor and at the end we eliminate right
away factors that no longer appear.
what type of algorithms are stepwise selection? Correct Answer-Greedy algorithms - at each step they
take one thing that looks best
what is LASSO Correct Answer-a variable selection method where the coefficients are determined by
both minimizing the squared error and the sum of their absolute value not being over a certain
threshold t
How do you choose t in LASSO Correct Answer-use the lasso approach with different values of t and see
which gives the best trade
[Show More]