Statistics for Sociologists/Chapter 6: Recoding and Transforming Variables

Chapter 6: Recoding and Transforming Variables Consider GOD, a GSS variable measuring belief in God, which — brace yourself — has response categories coded 1 through 6, plus 0, 8, and 98 sitting off to the side meaning "not applicable," "don't know," and "no answer." Six real categories. Three impostors. All nine of them stored as plain numbers, indistinguishable from one another until you go looking. What could go wrong? Several things, all at once, unless you know how to recode. This chapter is about reshaping variables into the form your analysis actually needs — collapsing categories that are too fine-grained, reversing scales that point the wrong direction, building new variables out of old ones, and, always, checking your work before you trust it. Why Recode at All? GSS variables were designed to capture rich detail, not to be immediately analysis-ready. A five-point happiness scale might need collapsing to a simpler happy/not-happy split for a specific research question. An age variable might be more useful as age groups than as exact years. A scale might run backward relative to how you want to interpret it. None of this means the original variable was poorly designed — it means the original variable was designed for flexibility, and recoding is how you spend some of that flexibility on the specific question in front of you. Collapsing Categories Sometimes a variable has more categories than your research question needs. GSS's HAPPY variable has three levels (very happy, pretty happy, not too happy); collapsing it to two (happy vs. not happy) trades some nuance for a simpler, often more interpretable comparison. Base R, with ifelse(): gss$HAPPY_binary <- ifelse(gss$HAPPY == 1, "Happy", "Not Happy") dplyr, with case_when(): library(dplyr) gss <- gss %>% mutate(HAPPY_binary = case_when( HAPPY == 1 ~ "Happy", HAPPY %in% c(2, 3) ~ "Not Happy", TRUE ~ NA_character_ )) Notice the case_when() version's final line, TRUE ~ NA_character_ — this is a deliberate safety net. Any value that doesn't match an earlier condition (including, importantly, any leftover missing codes you haven't yet recoded to NA) falls through to a genuine missing value instead of silently landing in whichever category happens to come first. ifelse() has no equivalent safety net built in; a value of 8 ("don't know") that slipped through uncaught would land in "Not Happy" purely because it isn't a 1, which is exactly the kind of quiet, plausible-looking error that makes it into a final paper unless you check. You can also use the car package to recode variables: car, with recode: library(car) gss$happy_binary <- car::recode(gss$happy, '1=1; 2:3=2; else=NA') Like was done with dplyr, the car package has an option for "everything else" that is not captured by the core code. In this case, everything coded as "1" stays a "1." Everything coded "2" or "3" becomes a "2." And the last part of the code tells R to make sure everything else is recoded as "NA." Creating Dummy Variables A dummy variable (also called an indicator variable) recodes any categorical variable into a 0/1 flag for a single category of interest — "is this respondent Protestant, yes or no" rather than "which of six religions is this respondent." Dummy variables are the standard currency of regression (Chapter 16), which generally wants 0s and 1s rather than unordered category labels. # Base R gss$is_protestant <- ifelse(gss$RELIG == 1, 1, 0) # dplyr gss <- gss %>% mutate(is_protestant = if_else(RELIG == 1, 1, 0)) # car gss$is_protestant <- car::recode(gss$RELIG, '1=1; else=0') Reversing Scales Occasionally a scale is coded in the opposite direction from how you want to interpret it — high numbers meaning "less" of something rather than "more." A common fix, for a scale running 1 to k, is: gss$SCALE_reversed <- (max(gss$SCALE, na.rm = TRUE) + 1) - gss$SCALE For a known 1-to-5 scale, this is often written more transparently as 6 - gss$SCALE — the general formula above just avoids hardcoding the "6" in case the scale's range changes later or you're writing a function meant to work on more than one scale. Whichever version you use, verify it: reverse-coding is invisible in the sense that the new variable looks exactly as plausible as the old one, right up until an analysis produces a relationship in the opposite direction from every published finding on the topic, and you spend an afternoon debugging a sign error that a two-line cross-tab would have caught in ten seconds. Creating New Variables From Existing Ones Continuous variables can be turned into meaningful categories — age groups from exact age, for instance: Base R, with nested ifelse(): gss$age_group <- ifelse(gss$AGE < 30, "Under 30", ifelse(gss$AGE < 50, "30-49", ifelse(gss$AGE < 65, "50-64", "65+"))) dplyr, with case_when() (considerably more readable once you have more than two cutoffs): library(dplyr) gss <- gss %>% mutate(age_group = case_when( AGE < 30 ~ "Under 30", AGE < 50 ~ "30-49", AGE < 65 ~ "50-64", AGE >= 65 ~ "65+", TRUE ~ NA_character_ )) This is the point in the book where case_when()'s advantage over nested ifelse() stops being subtle. Four nested ifelse() calls are already hard to read and easy to mis-parenthesize; a real analysis with eight or ten cutoffs in base R becomes an exercise in counting closing parentheses. case_when() scales to that complexity without becoming unreadable, which is a large part of why this book keeps introducing it as the dplyr default for anything beyond the simplest recode. car, with recode: library(car) gss$age_group <- car::recode(gss$age, '0:30=1; 31:49=2; 50:64=3; 65:99=4; else=NA') The car function doesn't convert the variable into a factor variable like the dplyr function does. Labeling Factors Once a categorical variable's numeric codes are finalized, converting it to a labeled factor makes it self-documenting — the same move from Chapter 4, applied here to your newly created variables: gss$age_group <- factor(gss$age_group, levels = c("Under 30", "30-49", "50-64", "65+")) Specifying levels explicitly, even when the labels are already correct character strings, matters for one specific reason: it controls the order categories appear in in tables and plots. Without it, R will default to alphabetical order — which is how you end up with a bar chart running "30-49, 50-64, 65+, Under 30" instead of the sensible chronological order a human reader expects. If you're working with a haven-imported, already-labelled variable (Chapter 3), the equivalent move is usually haven::as_factor() rather than typing out labels by hand — but for any variable you've constructed from scratch, like age_group above, you're the one supplying the labels, so the manual factor() approach is what you'll actually use. Another option is to use a package like labelled to add labels to your variable: library(labelled) # add variable label var_label(gss$HAPPY_binary) <- "HAPPY converted into two values" # add value labels val_labels(gss$HAPPY_binary) <- c("Happy"=1, "Not Happy"=2) Checking Your Work: table() and tabyl() Every recode should be verified, without exception, by cross-tabulating the old variable against the new one. This is not optional caution — it's the single fastest way to catch a recoding mistake before it propagates into every analysis downstream. Base R: table(gss$HAPPY, gss$HAPPY_binary) Using tabyl() from the janitor packageFirke, S. janitor: Simple Tools for Examining and Cleaning Dirty Data. R package. https://sfirke.github.io/janitor/ (a genuinely pleasant upgrade over base table() for this exact purpose): library(janitor) gss %>% tabyl(HAPPY, HAPPY_binary) Look at the resulting table and ask one question: does every original category land in the new category you intended, and nowhere else? For the GOD variable from this chapter's opening example, this check would immediately reveal whether your recode accidentally treated "don't know" (8) as a real belief category instead of routing it to NA — a mistake that a quick glance at the raw numbers would never catch, but a two-way table catches instantly. Open Science Corner #355C7D #FFFFFF #F8B195 Document every recode decision in your script with comments. "Collapsed HAPPY to binary because the three-category version had too few cases in 'not too happy' for a reliable comparison" is a sentence that takes ten seconds to type and will save you — or whoever reviews your work — twenty minutes of confused reverse-engineering six months from now. Future you, and any reviewer, will thank present you specifically for this habit. Chapter Summary Recoding reshapes variables into the form a specific analysis needs: collapsing fine-grained categories into simpler ones, building 0/1 dummy variables for regression, reversing scales that run the wrong direction, and constructing new categorical variables (like age groups) from continuous ones. case_when() in dplyr generally outperforms nested ifelse() in base R once more than a couple of conditions are involved, and both approaches benefit from an explicit fallback for unmatched or missing values. Once a recode is finalized, factor() (or haven::as_factor() for already-labelled data) makes categories self-documenting and controls their display order. None of this is trustworthy, however, without the last step: cross-tabulating old versus new with table() or tabyl() to confirm the recode did exactly what you intended — a habit that turns "I'm pretty sure this is right" into "I checked, and this is right." Practice Examples Example 1: Recode HAPPY Into a Binary Variable The Research Question: Is it more useful, for a given analysis, to treat happiness as a simple happy/not-happy split rather than three ordered categories? The Code (base R): gss$HAPPY_binary <- ifelse(gss$HAPPY == 1, "Happy", ifelse(gss$HAPPY %in% c(2, 3), "Not Happy", NA)) table(gss$HAPPY, gss$HAPPY_binary) The Code (dplyr): library(dplyr) gss <- gss %>% mutate(HAPPY_binary = case_when( HAPPY == 1 ~ "Happy", HAPPY %in% c(2, 3) ~ "Not Happy", TRUE ~ NA_character_ )) table(gss$HAPPY, gss$HAPPY_binary) The Output: Happy Not Happy 1 538 0 2 0 1184 3 0 317 The Interpretation: The cross-tabulation confirms the recode behaved exactly as intended: every respondent originally coded 1 ("very happy") landed in "Happy," and every respondent coded 2 or 3 landed in "Not Happy," with no crossover and no silent mishandling of missing codes. This binary version trades detail for simplicity — useful, for instance, going into Chapter 13's t-tests and Chapter 14's chi-square tests, both of which handle a clean two-category outcome more straightforwardly than a three-category one. Follow-Up Challenge: Would a different collapsing rule — say, treating "pretty happy" as "Happy" alongside "very happy," leaving only "not too happy" as "Not Happy" — change the story your data tells? Try it, and compare the resulting group sizes. Example 2: Create Age Groups From Continuous AGE The Research Question: Does grouping respondents into age categories reveal a story that raw continuous age doesn't make as immediately visible? The Code: library(dplyr) gss <- gss %>% mutate(age_group = case_when( AGE < 30 ~ "Under 30", AGE < 50 ~ "30-49", AGE < 65 ~ "50-64", AGE >= 65 ~ "65+", TRUE ~ NA_character_ ), age_group = factor(age_group, levels = c("Under 30", "30-49", "50-64", "65+"))) gss %>% count(age_group) The Output: age_group n Under 30 375 30-49 731 50-64 542 65+ 393 NA 3 The Interpretation: The sample skews toward middle age, with the two middle categories together accounting for over 60% of respondents. Note that the specific cutoffs chosen (30, 50, 65) are a judgment call, not a mathematical necessity — different cutoffs would produce a different-looking distribution from the exact same underlying data, which is worth remembering any time you encounter someone else's age groupings in a published table. Follow-Up Challenge: Try a different set of cutoffs (say, decade-based groups: 18-29, 30-39, 40-49, and so on) and compare the resulting frequency table. Does the overall pattern of "where most respondents fall" survive the change, or does it shift? Example 3: Reverse-Code a Scale and Verify The Research Question: If a scale variable is coded so that higher numbers mean less of the thing you care about, how do you correct that, and how do you confirm the correction worked? The Code: # Suppose SATJOB is coded 1 = very satisfied ... 4 = very dissatisfied, # and we want higher numbers to mean MORE satisfaction. gss$SATJOB_rev <- 5 - gss$SATJOB table(gss$SATJOB, gss$SATJOB_rev) The Output: 1 2 3 4 1 0 0 0 746 2 0 0 565 0 3 0 155 0 0 4 62 0 0 0 The Interpretation: The cross-tab confirms a clean, complete reversal — every original 1 maps to a new 4, every original 4 maps to a new 1, and so on, with no value left unmapped or duplicated across rows. This is exactly the check that would catch an off-by-one error (using 4 - SATJOB instead of 5 - SATJOB, say) before it embarrassingly reverses your entire results section. Follow-Up Challenge: Write out, in your own words, why the formula uses 5 - rather than 4 - for a 4-point scale. What would the general formula be for a scale running from 1 to k? References