33  Reproducibility and workflow

You finished an analysis six months ago, and now a reviewer wants you to re-run it with one variable removed. You open the project, hit “Source,” and nothing works: a package updated, a file path points to a folder that no longer exists, and you cannot remember whether you were supposed to run the cleaning script before or after the imputation script. The analysis produced correct results once. It will never produce them again.

Five tools prevent this: here, renv, Quarto, targets, and Git. Each one removes a specific way that analyses break, and this chapter gives each enough coverage to start using it today.

33.1 What reproducibility means

Same code plus same data should equal the same results. That is reproducibility, and if you squint, it looks like referential transparency at the project level: the analysis is an expression, and if it is reproducible you can replace it with its value, the results, without changing anything. A non-reproducible analysis is a function with hidden side effects; it depends on state you never declared.

Replicability asks a bigger question: same research question, new data, similar conclusions. Reproducibility is the lower bar, and most analyses fail it anyway. When Nature surveyed 1,576 researchers in 2016, more than 70% had tried and failed to reproduce another scientist’s experiment, and more than half had failed to reproduce their own. In 2022, Ana Trisovic and colleagues took more than 9,000 R files from over 2,000 replication datasets deposited at Harvard Dataverse between 2010 and 2020 and ran them: 74% crashed, and 56% still crashed after automated cleaning of the most common problems.

The state that never gets declared is the same handful of things every time: a file path hardcoded to "C:/Users/me/Desktop/data.csv", a package version nobody recorded (dplyr 1.0 behaves differently from dplyr 1.1), a manual step between scripts (“run this, then open Excel, then paste the table”), a random seed never set, and outputs stored apart from the code that produced them. Do not rely on discipline (“I will remember to run script 3 first”); build a system that makes the wrong thing hard. The first fix costs nothing to adopt.

TipOpinion

If your analysis requires a README that says “run these scripts in this order, but skip step 4 on Tuesdays,” it is not reproducible. It is a ritual.

33.2 Projects and file paths

Every hardcoded path assumes the directory structure will never change. An RStudio project (an .Rproj file) gives the project one root, and here::here() builds paths from it:

# Works regardless of where the script lives in the project
data <- read.csv(here::here("data", "penguins.csv"))

The call returns an absolute path on the fly, something like "C:/Users/me/analysis/data/penguins.csv", but you never type that path yourself. here finds the project root (the directory containing .Rproj, .here, or .git) and builds every path relative to it, so the same line works on any OS (forward slashes on Windows too), from any subdirectory, in Quarto documents, in test files, and in interactive sessions. It even works when you source() a script from a different directory, because it looks for the root, not for the script’s location.

setwd() and absolute paths both tie the code to one computer and one person’s folder layout; a source("C:/Users/me/Desktop/functions.R") line breaks on every other machine. A clean project layout looks like this:

my_analysis/
├── my_analysis.Rproj
├── data/               # raw data (read-only)
├── R/                  # functions
├── analysis/           # scripts or Quarto documents
├── output/             # results, figures, tables
└── renv.lock           # dependency lockfile

Raw data is sacred: never modify it. Read it, transform it in code, write outputs elsewhere. If someone asks “where did this number come from?”, you can trace it from the output file back through the code to the raw data, and that traceability breaks the moment code and data live in different places.

A common mistake is placing data-cleaning code in a different project from the analysis, so that reproducing the analysis requires finding and running a separate project first. Keep everything in one project, or use targets (Section 33.5) to formalize the dependency.

Another common mistake: using rm(list = ls()) at the top of a script to “clean up.” This clears your workspace but does not restart R, so hidden state (loaded packages, modified options, changed working directories) persists. Instead, restart R (Ctrl+Shift+F10 in RStudio) for a true clean slate.

TipOpinion

If you would not email your project folder to a collaborator and expect it to work, it is not organized well enough.

Exercises

  1. Create an RStudio project with the layout above. Place a CSV file in data/. Write a script in analysis/ that reads it using here::here(). Verify it works by opening the project fresh and running the script.
  2. Try here::here("data", "penguins.csv") from both the project root and a subdirectory. Does it return the same path?
  3. Open an old script of yours that uses setwd() or absolute paths. Rewrite it to use here::here(). Does it still work when you move the project to a different folder?

33.3 renv: locking dependencies

install.packages("dplyr") today gives you a different version than six months from now, and when your code breaks or your results silently change, nothing in your source files will explain why.

renv isolates your project’s package library:

renv::init()       # create a project-local library
renv::snapshot()   # record exact versions in renv.lock
renv::restore()    # install exactly the versions in the lockfile

init() gives the project a private library, so its package versions are separate from every other project on your machine. snapshot() writes renv.lock, a JSON file listing every package and its exact version. restore() reads that file and installs those versions, so a collaborator who clones your repository and runs it gets the same packages you used.

The workflow: init() once, snapshot() after installing or updating packages, commit renv.lock to Git. That is all. Switching between projects that use renv needs nothing special: each project has its own library, and renv activates it when you open the project. Packages installed in one project are invisible to another, so updating a package for one analysis cannot break a different one.

renv does not version R itself. If the R version matters, document it, or pin it with Docker or Nix (Section 33.9). The renv/ directory, which holds the installed packages, is normally git-ignored.

TipOpinion

renv.lock is boring. That is the point. Boring is reproducible.

Exercises

  1. Run renv::init() in a project. Install a package with install.packages(). Run renv::snapshot(). Open renv.lock and find the package and its version number.
  2. Delete the package from your library (simulate a fresh machine). Run renv::restore(). Verify the package is back.
  3. Open renv.lock in a text editor. What information does it store besides package names and versions?

33.4 Quarto: code and prose together

You have a script that produces three figures. You have a Word document that describes those figures. You update the script, re-run it, forget to update the Word document, and now the text describes figures that no longer exist. Code and prose live in different files, so they drift apart.

A Quarto document (.qmd) puts narrative text, code chunks, and their output in one file. When you render it, the code runs and the results appear inline:

---
title: "Penguin Analysis"
format: html
---

## Body mass by species

```{r}
library(palmerpenguins)
library(ggplot2)
penguins |>
  ggplot(aes(species, body_mass_g)) +
  geom_boxplot()
```

No separate script that makes the figures, no separate Word document that describes them. If the data changes, re-render and everything updates. To render the document, run from the terminal:

quarto render analysis.qmd           # defaults to the format in the YAML header
quarto render analysis.qmd --to pdf  # override the output format
quarto render analysis.qmd --to docx

The format field in the YAML header sets the default (html, pdf, docx, revealjs for slides), and the --to flag overrides it. HTML needs no extra software. PDF requires a LaTeX installation; quarto install tinytex handles that. Word output produces a .docx that collaborators who do not use R can read and comment on. In RStudio, the “Render” button (Ctrl+Shift+K) does the same thing without the terminal.

Quarto is the successor to R Markdown, and if you know one you mostly know the other. Quarto takes chunk options from #| lines instead of the chunk header, runs Python, Julia, and Observable alongside R, and has better defaults for academic output (cross-references, citations, callouts).

A program should be written for a person to read, with the machine’s version extracted from it afterwards. That was the argument a Stanford professor made in 1984, and the system he built to make it, WEB, paired Pascal with TeX so that one source file produced both the compiled program and a typeset explanation of it. The professor was Donald Knuth, whose TeX still typesets the PDF version of this book. His idea, literate programming, reached R as Sweave in 2002; knitr replaced Sweave in 2012 with more output formats and a cache; R Markdown wrapped knitr in a Markdown document; and Quarto (2022) generalized the same design across R, Python, Julia, and Observable.

Beyond single documents, Quarto renders presentations, websites, and books (this one included). Chunk options, placed at the top of each chunk with the #| prefix, control what the reader sees:

echo: false hides the code and shows only the output. eval: false shows the code but does not run it. fig-cap adds a caption. cache: true caches results so unchanged chunks do not re-run.

For academic writing, Quarto supports citations natively. Place a .bib file in your project and reference entries with @key:

---
title: "My Analysis"
bibliography: references.bib
---

As shown by @wickham2019, tidy data principles simplify analysis.

The citation is rendered in the output and a reference list is appended, so the reference section cannot drift from the citations. That keeps one document in sync with itself; an analysis spread over several long-running steps needs the same guarantee between the steps.

Exercises

  1. Create a .qmd file with a title, a code chunk that loads a dataset, and a code chunk that makes a plot. Render it to HTML. Change the data and re-render.
  2. Add echo: false to a code chunk. What changes in the rendered output?

33.5 targets: pipeline automation

Your analysis has five steps: clean data, fit model, run diagnostics, make figures, render report. You run them by hand, in order, every time. One day you change the model but forget to re-run the diagnostics. The figures now describe a model that no longer exists. How long before you notice?

targets asks you to write each step as a function in R/:

# R/functions.R
clean_penguins <- function(data) {
  data[!is.na(data$body_mass_g), ]
}

plot_results <- function(model) {
  plot(model, which = 1)
}

and to list the calls, with their inputs, in a file called _targets.R:

# _targets.R
library(targets)
source("R/functions.R")
tar_option_set(packages = c("dplyr", "ggplot2"))

list(
  tar_target(raw_data, read.csv(here::here("data", "penguins.csv"))),
  tar_target(clean_data, clean_penguins(raw_data)),
  tar_target(model, lm(body_mass_g ~ species, data = clean_data)),
  tar_target(fig, plot_results(model))
)

tar_make() runs the pipeline. tar_read(model) retrieves a cached result. tar_visnetwork() draws the dependency graph, with up-to-date targets in green, outdated ones in blue, and errored ones in red, which makes a stuck pipeline far easier to debug than a folder of numbered scripts.

Each target is a function call. Functions are the unit of computation (Chapter 7); targets are the unit of caching. If raw_data has not changed, clean_data does not re-run. If you change plot_results(), only fig re-runs. This matters most when a step is expensive: a model that takes an hour to fit is re-fit only when its code or its inputs change, so tweaking a color on a plot re-runs the plot in seconds off the cached model.

The caching works because the functions are pure: given the same inputs, clean_penguins() returns the same output, so targets can skip it when the inputs have not changed. A function that reads from a database, modifies a global variable, or depends on the current time would break the caching, because the same inputs would no longer guarantee the same output. Keeping each step in a named function in R/, instead of inline in the target list, also makes the steps testable on their own and lets the pipeline read like a table of contents.

Not every project needs targets. A single Quarto document handles simple analyses well. Reach for targets when you have long-running steps, many interdependent outputs, or pipelines that change often.

TipOpinion

targets is overkill for a homework assignment and necessary for a thesis chapter. Know where your project falls.

Exercises

  1. Define a small targets pipeline with three steps: read data, compute a summary, make a plot. Run it with tar_make(). Change the summary function and run tar_make() again. Observe which targets re-run and which are skipped.
  2. Run tar_visnetwork() on your pipeline. What do the colors mean?

33.6 Version control with Git

You have a working analysis. You try a new approach, and it breaks everything. You hit Ctrl+Z forty times, but the file is not quite back to where it was, and you are not sure which of the six files you changed. If only you had saved a snapshot before experimenting.

Git is that snapshot system. Every change is recorded as a commit: a frozen image of your project at one moment, with a message describing what changed and why. You can go back to any previous snapshot, compare any two, and branch your work into parallel lines that merge back together.

For an analysis, that buys you four things. Every commit is a checkpoint: a deleted function comes back with git checkout -- file.R, and when the analysis breaks, git diff shows exactly which lines differ from the last working state. Three months from now, when a reviewer asks why you removed a covariate, git log shows the commit where you removed it, with a message explaining the decision. Several people can work on the same project, each on their own branch. Git merges their changes and surfaces two edits to the same line as a conflict. A GitHub or GitLab repository also gives the project a permanent URL, and Zenodo can mint a DOI for a release from it, so the code can be cited.

The daily workflow

Git has many commands. You need six:

git status                              # what has changed?
git add file.R                          # stage a file for the next commit
git commit -m "Add bootstrap analysis"  # record the snapshot
git log --oneline                       # see recent history
git diff                                # see unstaged changes line by line
git push                                # send commits to GitHub

The mental model: you make changes to files, git add selects which changes to include in the next snapshot, git commit takes the snapshot with a message, and git push sends it to a remote server. git pull does the reverse, fetching commits from the remote and incorporating them into your local copy.

RStudio has a built-in Git pane that shows modified files, lets you stage changes, write commit messages, and push to GitHub without touching the terminal. For initial setup, usethis::use_git() initializes a repository, and usethis::use_github() creates a remote on GitHub and pushes your code in one step.

Branching

Branches let you try something without risking the main line of work:

git branch try-new-model     # create a branch
git checkout try-new-model   # switch to it
# ... make changes, commit ...
git checkout main            # switch back
git merge try-new-model      # incorporate the branch's changes

If the experiment works, merge it; if it does not, delete the branch. Either way the main branch is untouched.

What belongs in Git

Commit code, documentation, renv.lock, _targets.R, .gitignore, Quarto source files, and small data files (under a few MB). Leave out large data files, generated output (figures, HTML reports, the _targets/ cache), secrets and passwords, and the session litter R leaves behind: .Rhistory, .RData, .DS_Store.

.gitignore tells Git what to skip. A good starting point for R projects:

.Rhistory
.RData
.Rproj.user
_targets/
docs/
*.html
*.pdf

usethis::use_git_ignore() helps set it up.

Commit messages

Write commit messages that describe why you changed something, not what you changed. Git already records every added and deleted line; your message adds the reasoning that the diff cannot show.

Good: “Remove species interaction term (AIC worse by 4.2).” Bad: “Update analysis.R.” Good: “Fix off-by-one in bootstrap loop causing n+1 resamples.” Bad: “Bug fix.”

Your future self will search through git log when something breaks, and those messages are the only documentation of your decision-making process.

Going further

This section covers enough Git to track your work, collaborate, and recover from mistakes. It does not cover rebasing, cherry-picking, bisecting, or other advanced operations. Jenny Bryan’s Happy Git and GitHub for the useR (happygitwithr.com) is the best reference for R users and covers installation, SSH keys, merge conflicts, and common workflows in detail.

33.7 set.seed() and session info

Any analysis involving randomness (simulation, resampling, train/test splits) needs set.seed():

set.seed(42)
sample(1:100, 5)
#> [1] 49 65 25 74 18

Without set.seed(), every run produces different numbers. With it, the same seed always produces the same sequence. Place it at the top of your script or Quarto document; the specific number does not matter (42, 123, 2024, anything), but it must be fixed and documented.

One subtlety: R 3.6.0 changed how sample() draws from the generator, so a sample() call seeded under R 3.5 gives a different result under any later R with the same seed. To reproduce the old sequence, ask for the old method with set.seed(42, sample.kind = "Rounding"), and record the R version next to the seed in any case.

At the end of reports, record your environment:

sessionInfo()
# or
sessioninfo::session_info()

This captures R version, OS, and package versions. If results differ on another machine, the session info is the first place to look. Small practices, both of them, but they close gaps that the larger tools leave open.

Exercises

  1. Run the same sample() call twice without set.seed(). Do you get the same result? Now add set.seed(123) before each call. What changes?
  2. Run sessionInfo() and find: your R version, your operating system, and the version of a package you use frequently.

33.8 The reproducibility stack

An RStudio project with here costs nothing and belongs in everything, and Git belongs in anything that outlives a one-off exploration. The other layers arrive with the project’s needs, and one more sits beneath all of them: Docker or Nix, which pin R itself and the system libraries, and which the next section covers.

If your project… You need
Has any R code at all RStudio project + here
Will exist for more than a week Git
Uses packages that update renv
Produces a report or paper Quarto
Has steps that take >30 seconds targets
Must run on a different OS Docker or Nix

Each layer solves one failure mode. Together, they make “works on my machine” a non-issue.

33.9 Beyond renv: full-stack reproducibility

renv captures R package versions. It does not capture R itself, system libraries (libcurl, GDAL, libxml2), or the C compiler that built them. When your code depends on any of these (and spatial code, for instance, almost always depends on GDAL), renv alone leaves a gap.

Docker fills it by bundling everything into a container: OS, R, system libraries, packages, your code. The Rocker project (rocker-project.org) provides pre-built R images:

# Dockerfile
FROM rocker/r-ver:4.4.0
RUN install2.r dplyr ggplot2
COPY . /analysis
CMD ["Rscript", "analysis/main.R"]

A Dockerfile is a recipe that always produces the same environment. Build once, run anywhere. The trade-off is complexity: Docker adds a layer of tooling (images, containers, registries) that takes time to learn.

Nix, a package manager, treats every package as a pure function from its inputs (source code, dependencies, compiler flags) to its output (the built artifact). Fix all inputs and the output is deterministic, the same referential transparency that makes pure functions predictable in R.

The rix R package makes Nix accessible from R:

library(rix)

rix(
  r_ver = "4.4.0",
  r_pkgs = c("dplyr", "ggplot2", "palmerpenguins"),
  system_pkgs = NULL,
  ide = "rstudio",
  project_path = "."
)

This generates a default.nix file that pins everything: R version, package versions, system libraries, even the C compiler. Running nix-build on any machine with Nix installed produces an identical environment. Where renv.lock captures one layer (R packages), default.nix captures all of them.

Nix has a steep learning curve and runs natively only on Linux and macOS (Windows needs WSL); Docker is more widely used and runs everywhere. Both pin the full stack.

For most R users, renv is sufficient. Add Docker or Nix when your results depend on system-level components, when you need to guarantee reproducibility across operating systems, or when a collaborator reports “it doesn’t work on my machine” and the problem turns out to be something no R package can fix.