A First Course in Probability and Statistics By David Goldsman Paul Goldsman

A First Course in Probability and Statistics By David Goldsman Paul Goldsman Contents 1 Probability Basics 1 1.1 Introduction and Motivational Examples 1 1.2 Math Bootcamp . 4 1.2.1 The Joy of ...

A First Course in Probability and Statistics By David Goldsman Paul Goldsman Contents 1 Probability Basics 1 1.1 Introduction and Motivational Examples 1 1.2 Math Bootcamp . 4 1.2.1 The Joy of Sets 4 1.2.2 Calculus Primer . 7 1.2.3 Proving Things 13 1.3 Experiments and Probability Spaces 18 1.3.1 Sample Space . 19 1.3.2 Events . 19 1.3.3 What is Probability? . 20 1.3.4 Some Examples Involving Unions 21 1.4 Finite Sample Spaces . 24 1.5 Counting Techniques . 25 1.5.1 Baby Examples 25 1.5.2 Permutations . 27 1.5.3 Combinations . 28 1.6 Counting Applications 30 1.6.1 Hypergeometric Distribution 30 1.6.2 Binomial Distribution 31 1.6.3 Multinomial Coefficients . 31 1.6.4 Permutations vs. Combinations . 33 1.6.5 The Birthday Problem 34 1.6.6 The Envelope Problem 35 1.6.7 Poker Problems 35 1.7 Conditional Probability and Independence . 37 1.7.1 Conditional Probability . 38 1.7.2 Independence . 40 ix x CONTENTS 1.8 Bayes Theorem 43 1.9 Exercises . 45 2 Random Variables 51 2.1 Introduction and Definitions . 51 2.2 Discrete Random Variables . 53 2.3 Continuous Random Variables 55 2.4 Cumulative Distribution Functions . 58 2.5 Great Expectations 59 2.5.1 Expected Value 59 2.5.2 LOTUS, Moments, and Variance 61 2.5.3 LOTUS via Taylor Series 65 2.6 Moment Generating Functions . 67 2.7 Some Probability Inequalities 69 2.8 Functions of a Random Variable 71 2.8.1 Introduction and Baby Examples 72 2.8.2 Adolescent Inverse Transform Theorem Examples . 73 2.8.3 Grown-Up Honors Examples 75 2.9 Exercises . 77 3 Bivariate Random Variables 81 3.1 Introduction and Definitions . 81 3.1.1 Discrete Case . 81 3.1.2 Continuous Case . 82 3.1.3 Bivariate cdf’s 83 3.1.4 Marginal Distributions 84 3.2 Conditional Distributions 86 3.3 Independent Random Variables . 88 3.3.1 Definition and Basic Results 88 3.3.2 Consequences of Independence . 90 3.3.3 Random Samples . 93 3.4 Extensions of Conditional Distributions 94 3.4.1 Conditional Expectation . 94 3.4.2 Double Expectation . 95 3.4.3 Honors Applications . 96 3.5 Covariance and Correlation . 100 3.5.1 Basics . 100 CONTENTS xi 3.5.2 Correlation and Causation 103 3.5.3 A Couple of Worked Numerical Examples . 104 3.5.4 Additional Useful Theorems Involving Covariance . 105 3.6 Moment Generating Functions, Revisited . 106 3.7 Bivariate Functions of Random Variables . 109 3.7.1 Introduction and Basic Theory . 109 3.7.2 Examples . 110 3.8 Exercises . 112 4 Distributions 117 4.1 Discrete Distributions 117 4.1.1 Bernoulli and Binomial Distributions 117 4.1.2 Hypergeometric Distribution 118 4.1.3 Geometric and Negative Binomial Distributions 119 4.1.4 Poisson Processes and the Poisson Distribution 122 4.2 Continuous Distributions 126 4.2.1 Uniform Distribution . 126 4.2.2 Exponential, Erlang, and Gamma Distributions 126 4.2.3 Other Continuous Distributions . 130 4.3 The Normal Distribution and the Central Limit Theorem 132 4.3.1 Basics . 132 4.3.2 The Standard Normal Distribution . 135 4.3.3 The Sample Mean of Normal Observations 137 4.3.4 The Central Limit Theorem . 139 4.3.5 CLT Examples 141 4.4 Extensions of the Normal Distribution . 143 4.4.1 Bivariate Normal Distribution 143 4.4.2 Lognormal Distribution . 145 4.5 Computer Considerations 147 4.5.1 Evaluating pmf’s / pdf’s and cdf’s 147 4.5.2 Simulating Random Variables 148 4.6 Exercises . 150 5 Descriptive Statistics 155 5.1 Introduction to Statistics 156 5.1.1 What is Statistics? 156 5.1.2 Descriptive Statistics . 157 xii CONTENTS 5.1.3 Candidate Distributions . 160 5.2 Point Estimation . 161 5.2.1 Introduction to Estimation . 161 5.2.2 Unbiased Estimation . 162 5.2.3 Mean Squared Error . 165 5.2.4 Maximum Likelihood Estimation 166 5.2.5 Method of Moments . 172 5.3 Sampling Distributions 174 5.3.1 Normal Distribution . 174 5.3.2 χ2 Distribution 174 5.3.3 Student t Distribution 175 5.3.4 F Distribution 176 5.4 Exercises . 177 6 Confidence Intervals 181 6.1 Introduction to Confidence Intervals 182 6.2 Confidence Interval for Normal Mean (Variance Known) . 183 6.3 Confidence Interval for Difference of Normal Means (Variances Known)186 6.4 Confidence Interval for Normal Mean (Variance Unknown) 187 6.5 Confidence Intervals for Difference of Normal Means (Variances Unknown) 190 6.5.1 Variances Unknown but Equal . 191 6.5.2 Variances Unknown and Unequal 192 6.5.3 Paired Observations . 194 6.6 Confidence Interval for Normal Variance 196 6.7 Confidence Interval for Ratio of Normal Variances 197 6.8 Confidence Interval for Bernoulli Success Probability . 199 6.9 Exercises . 201 7 Hypothesis Testing 207 7.1 Introduction to Hypothesis Testing . 207 7.1.1 Our General Approach 208 7.1.2 The Errors of Our Ways . 210 7.2 Hypothesis Tests for Normal Means (Variance Known) 211 7.2.1 One-Sample Tests 212 7.2.2 Test Design 214 7.2.3 Two-Sample Tests 216 CONTENTS xiii 7.3 Hypothesis Tests for Normal Means (Variance Unknown) 217 7.3.1 One-Sample Test . 217 7.3.2 Two-Sample Tests 219 7.4 A Potpourri of Tests for Other Parameters 223 7.4.1 Normal Variance Test 224 7.4.2 Two-Sample Test for Equal Variances . 225 7.4.3 Bernoulli Proportion Test 226 7.4.4 Two-Sample Test for Equal Proportions 228 7.5 Goodness-of-Fit Tests 230 7.5.1 χ2 Goodness-of-Fit Test . 230 7.5.2 Beginner Examples 231 7.5.3 Mini-Project . 234 7.6 Exercises . 240 A Tables of Probability Distributions 245 B Quantile and cdf Tables 251