20 Function factories

Suppose you need twelve formatting functions: one for dollars, one for euros, one for percentages, one for each of nine other currencies your client cares about. You could write twelve functions by hand, each identical except for a prefix string. Or you could write one function that manufactures the other twelve. You have already built one of these: make_adder returns a function that remembers n, and Chapter 18 explained why the memory holds, because the returned function is a closure over the environment it was made in. This chapter builds whole families of functions from a single template.

20.1 The pattern

Start with one that raises numbers to a power:

power <- function(exponent) {
  function(x) x ^ exponent
}

square <- power(2)
cube <- power(3)

square(5)
#> [1] 25
cube(5)
#> [1] 125

When you call power(2), R creates an execution environment where exponent is 2, then returns an anonymous function that closes over that environment. square remembers exponent = 2 forever; cube remembers exponent = 3. Same logic, different parameters. power produces a family of related functions, and each member carries its own private copy of the parameter that distinguishes it from its siblings.

A function that returns a function this way is a function factory, and the functions it produces are closures.

Partial application and currying

power(2) fixes one argument of a two-argument operation and returns a function of the remaining argument: partial application. In lambda calculus, power is λe. λx. x^e, and applying it to 2 gives λx. x^2, the currying from Section 5.2. Haskell curries every function by default, so add 5 3 is really (add 5) 3, with add 5 returning a function. A function factory in R is currying done by hand.

Exercises

Write a function factory make_multiplier that takes a factor and returns a function that multiplies its argument by factor. Create double and triple and test them on the number 7.
What does power(1) return? Is it the identity function? Test it.
Write a factory make_greeter that takes a greeting string and returns a function that takes a name and produces the greeting. make_greeter("Hello")("Alice") should return "Hello, Alice".

20.2 The lazy evaluation trap

Here is a bug that costs an afternoon the first time you meet it:

exp <- 2
sq <- power(exp)
exp <- 3
sq(5)
#> [1] 125

That returns 125, not 25. R evaluates an argument the first time it is used, which can be long after the call, and until then it stores the unevaluated expression in an object called a promise (Section 23.1 has the full mechanism). Inside power(exp), exponent is a promise holding the expression exp. Nothing in the factory body uses exponent, so it stays a promise, and by the time sq(5) finally needs it, exp in the calling environment has become 3. The factory captured a promise, not a value. This is the lazy evaluation trap.

The loop version is worse:

fns <- list()
for (i in 1:3) {
  fns[[i]] <- function(x) x ^ i
}

fns[[1]](2)
#> [1] 8
fns[[2]](2)
#> [1] 8
fns[[3]](2)
#> [1] 8

All three return 8. By the time any of them is called, i is 3.

The fix is force():

power <- function(exponent) {
  force(exponent)
  function(x) x ^ exponent
}

force(exponent) evaluates the argument immediately, capturing the value rather than the promise, and now the factory works correctly:

exp <- 2
sq <- power(exp)
exp <- 3
sq(5)
#> [1] 25

25, as expected. The loop version has no argument to force: all three functions share the single i in the global environment. lapply() sidesteps the problem:

fns <- lapply(1:3, \(i) function(x) x ^ i)

fns[[1]](2)
#> [1] 2
fns[[2]](2)
#> [1] 4
fns[[3]](2)
#> [1] 8

Why does this work when the for loop didn’t? Because the anonymous function \(i) creates a new scope for each iteration. When lapply calls \(i) with the value 1, that call gets its own execution environment where i is 1, and the inner function(x) x ^ i closes over that environment. The next call gets a fresh environment where i is 2, and so on. Each manufactured function captures a different i in a different environment, whereas in the for loop there is only one i in one environment and every function points to it. lapply() also forces the argument before \(i) returns, so the promise problem from power(exp) cannot recur inside it.

Opinion

Every factory should force() every argument that the manufactured function uses. Make this a habit, not a debugging exercise. force() costs nothing, and the bug it prevents is invisible from the outside.

Exercises

Predict the output of the following code, then run it to check:

val <- 10
make_adder <- function(n) function(x) x + n
add_val <- make_adder(val)
val <- 20
add_val(1)

Fix the make_adder factory above with force(). Verify that changing val after creating add_val no longer affects the result.
Why does lapply(1:3, \(i) function(x) x ^ i) work without an explicit force() call?

20.3 Practical factories

The pattern shows up wherever the same logic repeats with different parameters. A formatter, for one, can have its prefix and suffix baked in:

make_formatter <- function(prefix, suffix = "") {
  force(prefix)
  force(suffix)
  function(x) paste0(prefix, x, suffix)
}

usd <- make_formatter("$")
pct <- make_formatter("", "%")

usd(42)
#> [1] "$42"
pct(0.15)
#> [1] "0.15%"

A filter can carry its cutoff:

above <- function(threshold) {
  force(threshold)
  function(x) x[x > threshold]
}

above_zero <- above(0)
above_zero(c(-2, 0, 3, -1, 5))
#> [1] 3 5

And a statistical transformation that depends on a single parameter is a natural factory. The Box-Cox transformation is log(x) when its parameter lambda is 0 and (x^lambda - 1) / lambda otherwise:

box_cox <- function(lambda) {
  force(lambda)
  if (lambda == 0) {
    \(x) log(x)
  } else {
    \(x) (x ^ lambda - 1) / lambda
  }
}

bc1 <- box_cox(1)
bc0 <- box_cox(0)

bc1(c(1, 2, 4))
#> [1] 0 1 3
bc0(c(1, 2, 4))
#> [1] 0.0000000 0.6931472 1.3862944

That if runs once, when the factory is called, and never again: the produced function does no branching at all. The choice of formula is paid for at construction time, and every later call to bc0 or bc1 pays only for the arithmetic. ggplot2 works the same way. scale_color_brewer(palette = "Set1") calls a factory, pal_brewer(), that returns a function mapping values to colors, so the palette is looked up once when the scale is built and the returned function only does the mapping. Separating the cost of choosing behavior from the cost of running it is called staged computation, and every factory you write does a version of it.

The next section starts from the other direction: instead of building a function from parameters, you take a function that already exists and modify its behavior.

Exercises

Write a factory between that takes low and high and returns a function that keeps only elements of a vector that fall in the range (low, high). Test it on 1:20 with bounds 5 and 15.
Write a factory make_counter that returns a function with no arguments. Each time the returned function is called, it should return the next integer (1, 2, 3, …). Hint: use <<- to modify a variable in the enclosing environment.

20.4 Memoization

Sometimes you already have the right function, and the only problem is speed:

library(memoise)

slow_square <- function(x) {
  Sys.sleep(1)
  x ^ 2
}

fast_square <- memoise(slow_square)

system.time(fast_square(10))
#>    user  system elapsed 
#>    0.00    0.00    1.03
system.time(fast_square(10))
#>    user  system elapsed 
#>    0.02    0.00    0.00

The first call takes about a second. The second is instant: memoise() remembered the argument and returned the stored result without running the body. Wrapping a function so that it caches its results this way is memoization. forget() clears the cache:

forget(fast_square)
#> [1] TRUE

A cache only pays off when inputs recur: expensive computations called repeatedly with the same arguments, API calls, simulation steps, recursive algorithms like Fibonacci that revisit the same subproblems, parsing the same file twice. It goes wrong on functions with side effects (a memoized plot is drawn once and never again), on functions whose inputs are large or new every time (the cache grows without bound), and on functions that depend on external state such as database contents or the system clock, where the cached answer goes stale.

Exercises

Write a function slow_sum that takes a vector, sleeps for 1 second, and returns the sum. Memoize it. Call it twice with the same input and verify the second call is fast.
What happens if you memoize rnorm? Try memo_rnorm <- memoise(rnorm); memo_rnorm(5); memo_rnorm(5). Is this useful?

20.5 Function operators

What do you do when a function might fail on some inputs, but you need to apply it to hundreds of values and cannot afford to let one error kill the whole pipeline? You could wrap every call in tryCatch. Or you could wrap the function itself once:

library(purrr)

safe_log <- safely(log)
safe_log(10)
#> $result
#> [1] 2.302585
#> 
#> $error
#> NULL
safe_log("a")
#> $result
#> NULL
#> 
#> $error
#> <simpleError in .f(...): non-numeric argument to mathematical function>

safely() took log and returned a new function that never errors; it returns a list with $result and $error instead. A function that takes a function and returns a modified function is a function operator: a factory whose raw material is a function rather than a number or a string.

possibly() is simpler: it returns a default value on error:

careful_log <- possibly(log, otherwise = NA)
careful_log(10)
#> [1] 2.302585
careful_log("a")
#> [1] NA

quietly() captures messages, warnings, and output as list components instead of printing them:

quiet_log <- quietly(log)
quiet_log(-1)
#> $result
#> [1] NaN
#> 
#> $output
#> [1] ""
#> 
#> $warnings
#> [1] "NaNs produced"
#> 
#> $messages
#> character(0)

You can write your own operators just as easily:

with_logging <- function(f) {
  force(f)
  function(...) {
    cat("Calling function with", length(list(...)), "argument(s)\n")
    f(...)
  }
}

logged_mean <- with_logging(mean)
logged_mean(1:10)
#> Calling function with 1 argument(s)
#> [1] 5.5

with_logging() calls force(f) for the same reason power() did. A function operator is a factory whose input happens to be a function, so the lazy evaluation trap from Section 20.2 applies: without force(), f is a promise, and if the variable it points to changes before the operator’s result is called, you get the wrong function.

Decorators

Python calls this wrapping a decorator and gives it syntax: @decorator above a function definition replaces the function with the wrapped version. The pattern is older than the syntax; it is one of the twenty-three in the Gang of Four’s Design Patterns (1994). R has no @, but safely(), memoise(), and with_logging() are all decorators: they change behavior by wrapping, leaving the original function untouched.

Exercises

Use safely() and map() to apply log() to the list list(1, -1, "a", 10). Extract the results and the errors separately.
Write a function operator with_timer that wraps a function so it prints the elapsed time each time it is called. Test it with Sys.sleep.
What does possibly(possibly(log, NA), NA) do? Is double-wrapping useful?

20.6 Composing factories and operators

Each of these patterns (factories, operators, functionals like map()) does one small thing, and they combine:

safe_log <- safely(log)
results <- map(list(1, -1, "a", 10), safe_log)
#> Warning in .f(...): NaNs produced

str(results)
#> List of 4
#>  $ :List of 2
#>   ..$ result: num 0
#>   ..$ error : NULL
#>  $ :List of 2
#>   ..$ result: num NaN
#>   ..$ error : NULL
#>  $ :List of 2
#>   ..$ result: NULL
#>   ..$ error :List of 2
#>   .. ..$ message: chr "non-numeric argument to mathematical function"
#>   .. ..$ call   : language .f(...)
#>   .. ..- attr(*, "class")= chr [1:3] "simpleError" "error" "condition"
#>  $ :List of 2
#>   ..$ result: num 2.3
#>   ..$ error : NULL

safely(log) is a function operator applied to log, and the result goes to map(). What if you need both safety and logging on the same function?

safe_logged_log <- with_logging(safely(log))
safe_logged_log(10)
#> Calling function with 1 argument(s)
#> $result
#> [1] 2.302585
#> 
#> $error
#> NULL
safe_logged_log("a")
#> Calling function with 1 argument(s)
#> $result
#> NULL
#> 
#> $error
#> <simpleError in .f(...): non-numeric argument to mathematical function>

The inner operator (safely) handles errors; the outer one (with_logging) adds logging. Each layer does one thing, and you combine them to get the behavior you want.

A factory builds specialized functions, an operator modifies them, and map() applies them across inputs.

formatters <- map(c("$", "EUR ", "GBP "), make_formatter)

map(formatters, \(f) f(100))
#> [[1]]
#> [1] "$100"
#> 
#> [[2]]
#> [1] "EUR 100"
#> 
#> [[3]]
#> [1] "GBP 100"

map() over a vector of prefixes produces a list of formatters, then map() over the formatters applies each one. A list of functions is data you can iterate over, filter, compose, and pass around just like a list of numbers. Chapter 21 uses that idea to collapse entire sequences into single values.