Statistics for Sociologists/Chapter 12: Confidence Intervals

Chapter 12: Confidence Intervals Chapters 10 and 11 built two pieces of machinery — the normal distribution and the standard error — that this chapter finally puts to practical use. A confidence interval is the first genuinely useful product of inferential statistics: not just a single number claiming to be "the" population mean, but an honest range that acknowledges you're estimating from a sample and quantifies exactly how much uncertainty that estimation carries. What a Confidence Interval Is (and Is Not) Here's the misconception first, because it's the one nearly everyone arrives with: a 95% confidence intervalNeyman, J. (1937). Outline of a theory of statistical estimation based on the classical theory of probability. Philosophical Transactions of the Royal Society of London. Series A, 236(767), 333–380. does not mean "there is a 95% probability the true population value falls inside this specific interval." That sounds like it should be true. It is not what a confidence interval claims. The correct interpretation is more roundabout: if you repeated your study many times, drawing a new sample and constructing a new 95% confidence interval each time, approximately 95% of those intervals — across all those hypothetical repetitions — would contain the true population value. Any one interval, the one you actually calculated from your actual sample, either does or doesn't contain the true value; there's no probability left to assign to it once it's calculated, any more than a coin already lying heads-up on the table has a "50% chance" of being heads. If you find this confusing, you are paying attention. This distinction trips up working researchers, not just students, and the honest thing to do is name that difficulty rather than paper over it with a simpler-sounding but wrong explanation. A useful analogy: a confidence interval is like a fishing net. You're not certain the fish (the true population parameter) is in any particular net you cast, but you know that 95% of nets cast the way yours was will, in fact, catch the fish. The confidence level describes the reliability of the method, across repeated use — not your certainty about this one specific result. Confidence Interval for a Mean The general form of a confidence interval is: For large samples, the critical value comes from the normal distribution (commonly written z*); for smaller samples, it comes from the t-distribution (t*), which is slightly wider — more conservative — to account for the extra uncertainty of estimating the population standard deviation from a small sample rather than knowing it outright. The quantity (critical value) × SE is called the margin of error — the "plus or minus" you see attached to any properly reported poll result. Three things affect a confidence interval's width, and each one makes intuitive sense once you say it out loud: a higher confidence level (99% instead of 95%) widens the interval, because being more certain requires casting a wider net; a larger sample size narrows it, because standard error shrinks with n (Chapter 11); and more variability in the underlying data widens it, because a noisier population is harder to pin down precisely regardless of sample size. Confidence Intervals in R Base R doesn't have an easy function to get a confidence interval. But there is a package that can be installed that makes this very easy: Rmisc. install.packages("Rmisc") library(Rmisc) CI((na.omit(gss$EDUC)), ci = 0.95) CI((na.omit(gss$EDUC)), ci = 0.99) The code above installs the "Rmisc" package, loads the library, then calculates a confidence interval for the variable "EDUC" in the "gss" dataset with a 95% confidence interval and then a 99% confidence interval. If you're a glutton for punishment, you can also do this manually with some math:x_bar <- mean(gss$EDUC, na.rm = TRUE) s <- sd(gss$EDUC, na.rm = TRUE) n <- sum(!is.na(gss$EDUC)) se <- s / sqrt(n) t_star <- qt(0.975, df = n - 1) # the t critical value for a 95% CI lower <- x_bar - t_star * se upper <- x_bar + t_star * se c(lower, upper) Note qt(0.975, ...) rather than 0.95 — a 95% confidence interval splits the remaining 5% evenly between the two tails, 2.5% on each side, which is why the critical value comes from the 97.5th percentile of the t-distribution rather than the 95th. The df = n - 1 should look familiar from Chapter 9's Sudoku explanation of degrees of freedom: calculating x_bar first spends one degree of freedom, leaving n − 1 for everything downstream, including the shape of the t-distribution used here. Visualizing Confidence Intervals Error bars are the standard way to show a confidence interval visually — a point for the estimate, with a line extending above and below to the interval's bounds. library(dplyr) library(ggplot2) ci_by_sex <- gss %>% group_by(SEX) %>% summarize( mean_age = mean(AGE, na.rm = TRUE), se = sd(AGE, na.rm = TRUE) / sqrt(sum(!is.na(AGE))), lower = mean_age - 1.96 * se, upper = mean_age + 1.96 * se ) ggplot(ci_by_sex, aes(x = factor(SEX), y = mean_age)) + geom_point(size = 3) + geom_errorbar(aes(ymin = lower, ymax = upper), width = 0.2) + labs(title = "Mean Age by Sex, with 95% Confidence Intervals", x = "Sex", y = "Mean Age") A genuinely useful habit worth building right now: when comparing two groups, look at whether their confidence intervals overlap. Widely separated, non-overlapping intervals are a strong visual hint of a real difference; heavily overlapping intervals suggest the groups may not differ meaningfully at all. This is an informal preview of Chapter 13's t-test — the formal version of exactly this comparison. Confidence Intervals vs. Hypothesis Tests: A Preview Confidence intervals and hypothesis tests (Chapter 13) are two views of the same underlying logic, and they will always agree with each other when applied consistently: if a 95% confidence interval for a difference between two groups does not contain zero, a hypothesis test on that same difference will reject the null hypothesis at α = 0.05, every time, by mathematical necessity rather than coincidence. Many working researchers actually prefer reporting confidence intervals over bare p-values for exactly this reason — a CI gives you the significance test's conclusion and a sense of the estimate's precision and plausible range, in one number instead of two. Practical Significance Revisited A confidence interval entirely above zero (or entirely below it) tells you the same directional story a significant p-value would — but it tells you more. It tells you the range of plausible effect sizes consistent with your data. A CI of [0.5, 12.3] and a CI of [4.8, 5.2] might both exclude zero and both count as "statistically significant," but they are telling very different stories about how precisely you've pinned down the true effect, and about how large that effect plausibly is. This is the same theme this book has returned to since Chapter 1: significance answers "is something going on here?"; the confidence interval, like the effect size, starts to answer "how much, and how sure are we?" Chapter Summary A confidence interval provides a range of plausible values for a population parameter, built from a sample estimate plus or minus a margin of error. The correct interpretation is about the reliability of the method across repeated sampling (95% of intervals built this way will contain the true value), not a probability statement about any one specific interval — a genuinely counterintuitive distinction worth sitting with rather than glossing over. Confidence interval width depends on confidence level, sample size, and underlying variability, and R's t.test() returns a 95% CI by default alongside its hypothesis test output. Visualizing CIs with error bars makes group comparisons intuitive, and non-overlapping intervals are an informal preview of statistical significance, formalized properly in Chapter 13. A confidence interval that excludes zero carries the same directional information as a significant p-value, plus something a p-value alone cannot offer: a sense of how large the effect plausibly is, and how precisely it's been estimated. Practice Examples Example 1: 95% CI for Mean EDUC The Research Question: What is a plausible range for the true population mean of years of education, based on this sample? The Code: t.test(gss$EDUC) The Output: One Sample t-test data: gss$EDUC t = 210.4, df = 2019, p-value < 2.2e-16 95 percent confidence interval: 13.55 13.81 sample estimates: mean of x 13.68 The Interpretation: We're 95% confident, in the repeated-sampling sense described earlier in this chapter, that the true population mean years of education for U.S. adults (circa 2010) falls between 13.55 and 13.81 years. Notice how narrow this interval is — a direct consequence of the GSS's large sample size, per Chapter 11's discussion of standard error shrinking with n. A smaller survey estimating the same quantity would produce a noticeably wider, less precise interval. Follow-Up Challenge: Compute the margin of error directly (half the interval's width). How does it compare to 1.96 times the standard error you'd calculate manually? Should they match closely? Example 2: CIs for Mean AGE by SEX, With Error Bars The Research Question: Do men and women in the sample have meaningfully different average ages, based on whether their confidence intervals overlap? The Code: library(dplyr) library(ggplot2) ci_by_sex <- gss %>% group_by(SEX) %>% summarize( mean_age = mean(AGE, na.rm = TRUE), se = sd(AGE, na.rm = TRUE) / sqrt(sum(!is.na(AGE))), lower = mean_age - 1.96 * se, upper = mean_age + 1.96 * se ) ci_by_sex ggplot(ci_by_sex, aes(x = factor(SEX), y = mean_age)) + geom_point(size = 3) + geom_errorbar(aes(ymin = lower, ymax = upper), width = 0.2) + labs(title = "Mean Age by Sex, 95% CI", x = "Sex", y = "Mean Age") The Output: SEX mean_age se lower upper 1 1 46.1 0.45 45.22 46.98 2 2 47.6 0.42 46.78 48.42 The Interpretation: The two confidence intervals overlap substantially (45.22–46.98 versus 46.78–48.42), which suggests any difference in average age between men and women in this sample, if real at all, is small relative to the uncertainty in both estimates. This is an informal read; Chapter 13's independent-samples t-test will give you a formal answer to the same question. Follow-Up Challenge: Compute the same comparison for EDUC by SEX instead of AGE. Do those intervals overlap as much, or does one group's interval sit clearly apart from the other's? Example 3: Changing the Confidence Level The Research Question: How does the width of a confidence interval change as you demand more or less confidence? The Code: t.test(gss$EDUC, conf.level = 0.90) t.test(gss$EDUC, conf.level = 0.95) t.test(gss$EDUC, conf.level = 0.99) The Output: 90 percent CI: 13.58 13.78 95 percent CI: 13.55 13.81 99 percent CI: 13.49 13.87 The Interpretation: Moving from 90% to 99% confidence widens the interval at both ends — you're demanding a net that catches the true value more reliably across repeated sampling, and a more reliable net has to be wider. There is no free lunch here: more confidence always costs precision, for a fixed sample size. The only way to get both more confidence and a narrower interval simultaneously is a larger sample, which is exactly the tension every study designer has to navigate. Follow-Up Challenge: Predict, before running the code, whether an 80% confidence interval would be narrower or wider than the 90% one above. Confirm your prediction with conf.level = 0.80. References