23  Lazy evaluation

You write filter(penguins, species == "Adelie") and it works. But species isn’t a variable in your environment. How does R know you mean a column? The answer starts with laziness.

R doesn’t evaluate function arguments when you pass them. It waits. Arguments are wrapped in invisible objects called promises, and those promises sit unevaluated until someone actually needs their value. This small design choice has enormous consequences: it enables default arguments that depend on other arguments, functions that capture your expressions instead of evaluating them, and the entire tidyverse style of writing column names without quotes.

The idea has a history. In the early days of programming languages, there were two camps. Fortran and C evaluated arguments before passing them (call-by-value): compute the result, hand it over. Algol 60 (1960) tried call-by-name: pass the expression itself, re-evaluate it every time the function uses it. Call-by-name turned out to be expensive and confusing. In 1976, Dan Friedman and David Wise, and independently Peter Henderson and James Morris, proposed a compromise: pass the expression unevaluated, but cache the result after the first evaluation. Evaluate at most once, not eagerly, not repeatedly. They called it lazy evaluation. Haskell (1990) made laziness the default for everything. R took a narrower path: function arguments are lazy, everything else is eager. This middle ground, inherited through Scheme and S, gives R the power of expression capture without the performance puzzles of pervasive laziness.

This chapter traces the path from promises to { }. You will understand how R’s laziness powers non-standard evaluation, why data masking feels like magic, and how to write functions that pass column names through to dplyr. Full metaprogramming is in Chapter 26; here, the goal is understanding the mechanism well enough to use it.

23.1 Promises

When you call f(x + 1), R does not compute x + 1 and pass the result. It wraps x + 1 in a promise: a bundle containing the expression (x + 1) and the environment where that expression should be evaluated (your calling environment). The promise travels into f unevaluated.

The first time f uses that argument, R evaluates the promise: it takes the expression, evaluates it in the stored environment, and caches the result. If the argument is never used, the promise is never evaluated. If it’s used twice, the second access returns the cached value, not a re-evaluation.

A promise has three components:

  1. The expression: what was written (x + 1).
  2. The environment: where to evaluate it (the caller’s environment).
  3. The cached value: filled in after the first evaluation, empty before.

Promises are invisible by design: R has no user-facing “promise” type. You cannot create one, inspect one with str(), or test with is.promise(). The invisibility is deliberate, because inspecting a promise would force it, changing program behavior. The abstraction only works because you cannot look behind it.

In the terminology of lambda calculus, this is call-by-need evaluation: the argument is passed unevaluated (like call-by-name) but the result is cached after the first evaluation (unlike call-by-name, which would re-evaluate each time). Lambda calculus defines three evaluation strategies: call-by-value (strict, as in C and Python), call-by-name (non-strict, as in Algol 60), and call-by-need (lazy, as in Haskell). R is mostly call-by-value but uses call-by-need for function arguments via promises. The Church-Rosser theorem guarantees that if a reduction terminates, all strategies reach the same normal form, but they may differ in whether they terminate at all.

This is why default arguments can refer to other arguments:

f <- function(x, y = x * 2) y
f(5)
#> [1] 10

The default for y is a promise containing the expression x * 2. When R needs y’s value, it evaluates x * 2 inside the function body, where x is already 5. The result is 10. If you supply y explicitly, the default promise is never evaluated:

f(5, 99)
#> [1] 99

Defaults can even depend on computations that happen inside the function:

g <- function(x, n = length(x)) n
g(c(1, 2, 3))
#> [1] 3

The default n = length(x) is an expression, not a value. It’s evaluated when n is needed, by which time x is already bound. This would be impossible if R evaluated defaults at the moment of the call.

Exercises

  1. Predict the output of this code, then run it:

    h <- function(x) {
  cat("about to use x\n")
  x
}
h(cat("evaluating argument\n"))

    Which cat() call prints first? Why?

  2. Write a function f(x, y = x + 1) and call f(10). What is y? Now call f(10, 50). What changed?

  3. What happens if you call a function that never uses its argument?

    ignore <- function(x) 42
ignore(stop("this should error"))

    Does it error? Why or why not?

23.2 Consequences of laziness

Lazy evaluation has several consequences that surprise people coming from other languages.

In R, only function arguments are lazy; everything else is eager. Haskell took the opposite approach: everything is lazy. let xs = [1..] creates an infinite list without computing all elements, enabling infinite streams and on-demand computation, but making reasoning about performance harder. R chose a middle ground: arguments are lazy (enabling NSE and flexible defaults), the rest is eager.

Side effects in arguments may never happen. If a function doesn’t use an argument, the argument’s expression is never evaluated. That means side effects (printing, writing files, raising errors) inside the expression never occur:

quiet <- function(x) "I ignore my argument"
quiet(print("you will never see this"))
#> [1] "I ignore my argument"

No output from print(). The promise was never forced.

missing() can detect unsupplied arguments. Because arguments start as unevaluated promises, R can tell the difference between “the caller supplied this argument” and “the caller didn’t.” The missing() function checks this:

report <- function(x) {
  if (missing(x)) "not supplied" else "supplied"
}
report()
#> [1] "not supplied"
report(42)
#> [1] "supplied"

This only works because promises are inspectable before evaluation. Once an argument is used, missing() is no longer meaningful.

Default expressions, not default values. A function like function(x, n = length(x)) does not store a default value for n. It stores a default expression. That expression is evaluated lazily, inside the function, at the moment n is first accessed. This means defaults can depend on other arguments, on computations that happen earlier in the function body, or on variables in the function’s environment.

The lazy evaluation trap in function factories. If you read Chapter 20, you saw that promises can cause trouble when a function factory captures a loop variable:

funs <- list()
for (i in 1:3) {
  funs[[i]] <- function() i
}
funs[[1]]()
#> [1] 3
funs[[2]]()
#> [1] 3

All three functions return 3 because i was captured as a promise, and by the time any function is called, the loop has finished and i is 3. The fix is force(), which evaluates a promise immediately. But force() alone is not enough: you also need local() to create a fresh environment for each iteration, so that each function captures its own copy of i rather than sharing the loop variable:

funs <- list()
for (i in 1:3) {
  local({
    force(i)
    funs[[i]] <<- function() i
  })
}
funs[[1]]()
#> [1] 3
funs[[2]]()
#> [1] 3

local() evaluates its body in a new, temporary environment. Each iteration gets its own environment with its own i. force(i) inside that environment evaluates the promise immediately, locking in the current loop value. Without local(), all three functions would still share the same i binding. Without force(), the promise might not be evaluated until after the loop finishes. You need both: local() for isolation, force() for immediate evaluation.

force(x) is just x, nothing more. It accesses the argument, which triggers evaluation and caching. The name communicates intent: “evaluate this now, don’t wait.”

Exercises

  1. Predict: does quiet(log(-1)) produce a warning? Why or why not? (Use the quiet function defined above.)

  2. Write a function greet(name = "stranger") that returns paste("Hello,", name). Call it with and without an argument. Explain which default mechanism makes this work.

23.3 What is non-standard evaluation?

Standard evaluation is simple: R evaluates an expression, gets a value, and passes the value to a function. When you write mean(c(1, 2, 3)), R evaluates c(1, 2, 3) to produce a numeric vector, then passes that vector to mean. The function never sees the expression c(1, 2, 3). It only sees [1] 1 2 3.

Non-standard evaluation (NSE) is different. The function captures the expression before evaluating it, then decides what to do with it. Consider subset():

df <- data.frame(x = 1:5, y = c(10, 20, 30, 40, 50))
subset(df, x > 3)
#>   x  y
#> 4 4 40
#> 5 5 50

You wrote x > 3, but x isn’t a variable in your global environment. It’s a column of df. How did subset know? Because subset doesn’t evaluate x > 3 in your environment. It captures the expression using substitute(), then evaluates it inside df using eval():

# Simplified subset internals
subset.data.frame <- function(x, subset, ...) {
  expr <- substitute(subset)
  row_mask <- eval(expr, x, parent.frame())
  x[row_mask & !is.na(row_mask), , drop = FALSE]
}

substitute() grabs the unevaluated promise. eval(expr, x) evaluates it in an environment where the columns of x are visible as variables. That’s why x > 3 finds the column x instead of looking for a variable named x in your workspace.

This is the core of NSE: intercept the expression, choose where to evaluate it. The benefit is convenience. Compare:

# Standard evaluation: explicit, verbose
df[df$x > 3, ]

# Non-standard evaluation: concise, readable
subset(df, x > 3)

NSE is the reason the tidyverse feels like a domain-specific language for data analysis. filter(penguins, species == "Adelie") reads like a sentence because you write column names directly, without $ or quotes. The cost is that the rules are less obvious: the meaning of species depends on which function you’re inside, not just on what’s in your environment.

Exercises

  1. Run subset(df, x > 3) with the data frame above, then try df[df$x > 3, ]. Verify they produce the same result. Which is easier to read?

  2. What happens if you define x <- 100 in your global environment and then run subset(df, x > 3) again? Does it use the column or the variable? Why?

23.4 Data masking

Data masking is the tidyverse’s name for NSE applied to data frames. When you write:

library(dplyr)
filter(penguins, species == "Adelie")

the column species is looked up first in penguins, then in your calling environment. The data masks the environment. If the data frame has a column called species, that column wins over any variable named species in your workspace.

This is why aes(x = bill_length_mm) works in ggplot2 without quotes. ggplot2 captures the expression and evaluates it against the data at plot time. The data frame’s columns become temporarily available as if they were variables.

The benefit is enormous for interactive analysis. You type column names hundreds of times in a session. Not having to write penguins$species or penguins[["species"]] each time saves keystrokes and keeps your code readable.

The cost appears when you try to program with data-masked functions. Suppose you want to write a function that filters by a column whose name is stored in a variable:

col <- "species"
filter(penguins, col == "Adelie")

This doesn’t work. filter looks for a column named col in penguins, doesn’t find one, then finds col in your environment (the string "species"), and compares the string "species" to "Adelie". That’s not what you wanted.

To see the mechanism concretely, here is a simplified filter() that you could write yourself in about ten lines:

my_filter <- function(.data, expr) {
  e <- substitute(expr)                    # capture the caller's expression
  env <- list2env(.data, parent = parent.frame())  # build an environment from the data frame
  mask <- eval(e, envir = env)             # evaluate the expression in that environment
  .data[mask & !is.na(mask), , drop = FALSE]
}

df <- data.frame(x = 1:5, y = c(10, 20, 30, 40, 50))
my_filter(df, x > 3)
#>   x  y
#> 4 4 40
#> 5 5 50

substitute(expr) captures the caller’s expression (x > 3) without evaluating it, exactly as a promise holds an unevaluated expression (Section 23.1). list2env(.data, parent = parent.frame()) creates a new environment whose bindings are the columns of the data frame, with the caller’s environment as the parent (so variables not in the data frame are still found). eval(e, envir = env) forces the expression in that environment, where x resolves to the column, not to anything in the global environment. The entire mechanism rests on the laziness you learned at the start of this chapter: arguments arrive as unevaluated promises, and NSE exploits that window between arrival and evaluation to inspect and redirect them. This is, in essence, what dplyr::filter() does, with more machinery for tidy evaluation, error handling, and grouped data frames layered on top.

Data masking is optimized for the common case (interactive analysis) at the expense of the less common case (programming). The tidyverse’s answer to this trade-off is tidy evaluation.

Exercises

  1. Define x <- 1000 in your global environment. Create a data frame df <- data.frame(x = 1:5). What does dplyr::filter(df, x > 3) return? Does it use the column or the variable?

  2. Explain in one sentence why aes(x = bill_length_mm) doesn’t need quotes around bill_length_mm.

23.5 Tidy evaluation basics

The embrace operator { } (called “curly-curly”) solves the most common programming-with-NSE problem: passing a column name through a function to a dplyr verb.

library(dplyr)

my_summary <- function(data, var) {
  data |>
    summarise(mean = mean({{ var }}, na.rm = TRUE))
}

{ var } says: “take whatever the caller passed as var, capture it as a data-masked expression, and inject it here.” The caller writes column names unquoted, just like they would with dplyr directly:

my_summary(penguins, body_mass_g)

Without { }, you’d have to teach your users a different syntax for your wrapper than they use for dplyr itself. With it, the wrapper is transparent.

For tidy selection (used in select(), across(), and similar), { } also works:

my_select <- function(data, cols) {
  data |> select({{ cols }})
}
my_select(penguins, starts_with("bill"))

The caller passes tidy-select expressions the same way they would to select() directly. The function just forwards them.

When { } isn’t enough, there are more tools. .data[["col_name"]] lets you use string column names inside data-masked functions. !! (bang-bang) and enquo() give finer control over quoting and unquoting. These are part of the full tidy evaluation framework in the rlang package, and they matter when you’re building complex programmatic interfaces. But they belong in Chapter 26, not here.

TipOpinion

For 80% of programming-with-dplyr tasks, { } is all you need. Learn it, use it, and don’t go deeper until you have to. The full tidy evaluation system (quosures, enquo(), !!, !!!) exists but is rarely needed. Most wrapper functions never need it.

Exercises

  1. Write a function my_count(data, group_var) that uses { } to count rows per group. Test it with any data frame.

  2. Write a function my_arrange(data, sort_var) that arranges a data frame by a column passed by the caller. Use { }.

  3. What happens if you forget { } and write summarise(data, mean = mean(var, na.rm = TRUE)) inside a wrapper function? What error do you get?

23.6 The trade-offs of NSE

NSE is a design trade-off, and it’s worth being honest about both sides.

Convenience vs. predictability. filter(df, x > 5) is concise. But what if there’s a variable x in your environment and a column x in your data frame? Data masking means the column wins. That’s usually correct, but when it’s not, the bug is subtle: no error, just wrong results.

Interactive vs. programmatic. NSE is optimized for the console. You type select(df, name, age) and it just works. But writing a function that wraps select requires { }, which is an extra concept to learn. The interactive user pays nothing; the programmer pays a tax.

Debugging. Errors inside data-masked functions can be cryptic. “Object ‘species’ not found” might mean you forgot to load the data, misspelled a column name, or called a data-masked function outside its expected context. The error message doesn’t distinguish these cases.

This isn’t a tidyverse invention. Base R uses NSE in subset(), with(), transform(), and the formula interface (lm(y ~ x, data = df)). The tidyverse uses it more systematically, but the idea is as old as S.

TipOpinion

NSE is the right trade-off for data analysis. You type column names hundreds of times a day, and quoting each one would be painful. The cost is paid when programming (writing reusable functions), not when analyzing (exploring data interactively). That’s a good trade: most R users spend more time analyzing than programming. Accept the trade-off, learn { } for when you cross the line into programming, and move on.