Computer Science > STUDY GUIDE > Georgia Institute Of Technology - ISYE 6501quiz2-study_guide. (All)

Georgia Institute Of Technology - ISYE 6501quiz2-study_guide.

Document Content and Description Below

# 11.1: Introduction to Variable Selection + factorbased models: classification, clustering, regression + why limit number of factors? 1. avoid overfitting: especially problematic when # of facto... rs is => # of data points; cauases model to fit to random effects 2. simplicity: + also implies less data collection is required + reduces chances of identifying insignificant factors + easier to interpret and communicate results (i.e. for prescriptive analysis) + sometimes factors can be illegal to use (e.g. race, gender, religion, etc.), so must be careful not to use corresponding highly correlated factors; also, it can be difficult to prove that an overly complex model avoids using these factors + check: simpler models with fewer factors avoids (1) overfitting and (2) difficulty of interpretation (not necessarily low prediction quality nor bias in the most important factors) # 11.2: Models for Variable Selection + forward selection, backward elimination, and stepwise regression + "greedy": takes action without considering future options + criteria other than pvalue can be used to determine factor importance (e.g. R^2, AIC, BIC) + lasso: + adds constraint to standard regression optimization objective of minimizing sum of scquared errors by adding a "budget" tau to use for sum of coefficients (i.e. adds constraint to minimize size of coefficients, which is similar to SVM) + data must be scaled beforehand + some coefficients may be calculated to be 0 + how to choose tau? try different values and + consider number of variables + consider quality of model + ridge regression: + similar to lasso, but constrains on a combination of the absolute value of the coefficients and their squares + same issues regarding scaling and choosing parameters (in this case, tau and lambda) + ridge regression: [Show More]

Last updated: 2 years ago

Preview 1 out of 10 pages

Buy Now

Instant download

We Accept: