Skip to contents

Choosing Between getaca and the Alternatives

Source: vignettes/alternatives.Rmd
alternatives.Rmd

Several established approaches solve neighbouring problems, and for a good number of packages one of them is the better fit. This article describes what each is built around and where the boundaries fall, so the choice can be made on the shape of the problem rather than on which package someone met first.

The question getaca is built around: a package needs a file it cannot ship, and every machine running that package should get the same bytes.

Bundling the data in the package

The first thing to rule in or out. A dataset in data/ needs no cache, no network, no checksum and no policy, and every one of those is a moving part that can fail.

CRAN’s size guidance makes this a question of megabytes rather than gigabytes, and the release cadence question usually settles it before the size does: data in data/ release when the package releases. For a reference table that changes yearly and a package that releases yearly, coupling them is a feature.

Bundle when the data are small, static relative to your release cycle, and redistributable. Look further when any of the three fails.

A companion data package

The classic R answer to data that are too large for the main package but still manageable: ship them as their own package, list it in Suggests, and put it on CRAN, Bioconductor or r-universe.

companion data package getaca
Size fits a repository too large to bundle
Release cadence coupled to code releases independent of them
Shape naturally R objects any file, any format
Granularity all of it, always users take what they need
License redistribution permitted download permitted, redistribution discouraged
Installation the usual package machinery first use, or an explicit prefetch
Offline works once installed works once cached

The licence row is the one that decides it most often. A companion package redistributes the data, which needs terms that permit redistribution. Plenty of scientific datasets permit download and discourage or forbid redistribution, and for those a companion package is not available whatever its other merits.

Granularity is the second. A package needing one of fifteen reference files, at a gigabyte each, would install fifteen through a companion package. Under getaca a user takes what they use.

A companion package can itself use getaca, which is occasionally the right structure when the companion owns expensive build logic. It is rarely the first recommendation, since it moves the complexity rather than removing it.

pins

pins publishes “data sets, models, and other R objects, making it easy to share them across projects and with your colleagues”, across boards including local folders, Posit Connect and AWS S3.

The unit is a pin on a board, and the person at the centre of the design is the one publishing an artefact for others to read. Boards abstract over where that artefact lives, which is what makes the same code work against a network drive during development and Posit Connect in production.

Reach for pins when the artefact is yours, the audience is a team or an organisation, the storage backend matters, and writing is part of the workflow. It covers a range of backends getaca has no interest in growing.

Reach for getaca when the artefact is a third party’s, the audience is everyone who installs your package, and the declaration has to travel inside the package so that R CMD check and an offline user both behave.

The two coexist. A package can declare its public reference data through getaca and its team’s internal model artefacts through pins without either knowing about the other.

BiocFileCache

BiocFileCache “creates a persistent on-disk cache of files that the user can add, update, and retrieve”, for resources that are costly or difficult to create and for web resources used across sessions, backed by an SQLite metadata database.

It is the established answer inside Bioconductor, and a Bioconductor package already carrying that dependency stack has little reason to add another. The model is a cache the calling package manages: your code decides what to add, when a cached copy needs updating, and what identity a resource has.

getaca makes those decisions from a declaration instead. Identity is package / name / version, resolution runs through a policy, and the retention sweeps are the package’s rather than the caller’s. That is a narrower contract, and it exists because the same behaviour then holds for every declaring package rather than being reimplemented per package.

Reach for BiocFileCache when you are in the Bioconductor ecosystem, when the resources are ones your code creates rather than ones a publisher versions, or when you want the cache under your own control.

pooch

The nearest equivalent outside R, and the one a reader arriving from Python will already know. pooch, “a friend to fetch your data files”, is where a Python package puts this problem: a registry of file names and hashes, a cache folder under the OS convention, downloaders and processors around them.

The overlap is the middle of the problem, fetch and hash and cache. The two ends differ, and both differences are about where the constraints come from.

At the front, a pooch registry is a registry.txt shipped as package data, and version is documented as “the version string for your project”, naming the subfolder the cache uses. Data identity is the declaring project’s identity, so a repaired mirror or a fresh upstream cut reaches users when the code does. A getaca declaration versions the data separately, holds several versions at once, names the head, and can resolve through a remote registry the author keeps, which is what lets 2026-09 reach an installed copy between releases. urls in pooch sets one URL per file; a getaca record takes a list of mirrors and walks it until one answers. pooch also accepts a doi: URL and asks figshare, Zenodo or Dataverse for the download location as it fetches. getaca asks the same three archives in registry_draft(), when the registry is written, and a record then carries the DOI as what the bytes are cited as and the locations as themselves.

At the back, the constraints are CRAN’s. R CMD check runs the tests, examples and vignettes of every package on machines with no network, and tools::R_user_dir() is permitted on condition its contents are actively managed. The check clamp, the three access helpers and the retention sweeps exist for those two sentences.

A downloader and a cache directory

The common alternative in practice: download.file() or curl, a directory under tools::R_user_dir(), and a file.exists() check. It is a few dozen lines and it works.

Two things cost more work than they look. The first is what happens when the bytes at the URL differ from the bytes you expected, which covers a truncated transfer, a proxy serving an HTML error page, and a publisher recutting a file. Each of those reaches your parser as a confusing error some distance from its cause.

The second is CRAN’s requirement to fail gracefully with an informative message when a resource is unavailable, across tests, examples and vignettes, on a machine with no network. That is the part that turns a few dozen lines into a few hundred, and it has to be written again in every package that depends on external data.

getaca is that layer written once. The declaring package supplies a registry.

Targets and the workflow tools

targets and drake manage a computational pipeline, tracking which steps need to re-run when inputs change. That is a different question from where a file comes from and whether it is the right file.

They compose. A targets pipeline can have a target whose command is getaca("backbone", package = "yourpkg"), which gives the pipeline a path, and the path a provenance record:

library(targets)

list(
  tar_target(backbone_path, getaca::getaca("backbone", package = "yourpkg"),
             format = "file"),
  tar_target(backbone, read_backbone(backbone_path)),
  tar_target(summary, summarise_backbone(backbone))
)

format = "file" makes targets watch the returned path, so the pipeline invalidates downstream targets when the resolved version changes. That is the combination worth reaching for: getaca decides which bytes, targets decides what to recompute.

renv

renv records which package versions an analysis used. A getaca pin records which data versions those packages resolved to. Neither subsumes the other: a package version does not determine a data version once the package is on the current policy, and a data version says nothing about the code that read it.

renv::snapshot()
getaca_pin(c("yourpkg", "otherpkg"))

Commit both files. Restoring is renv::restore() followed by getaca_prefetch() on a connected machine, after which the analysis runs offline. See vignette("policies") for what a pin holds.

Decision summary

If then
the data are small, static and redistributable put them in data/
the data are moderate, redistributable and release with your code a companion data package
you are publishing your own artefacts to a team pins
you are inside Bioconductor, or caching things your code creates BiocFileCache
you are managing which steps re-run targets, with getaca supplying the path
your package declares a third party’s versioned file it cannot ship getaca

What getaca deliberately does not do

Naming the boundary is part of choosing. None of these is planned:

  • Credential storage. A declaration names the environment variable a host requires, and getaca reads it at the moment of the request. It holds no credentials, reads no .netrc and talks to no keyring. See vignette("declaring").
  • Cloud storage abstractions. No S3, no Azure, no board concept. A URL is a URL.
  • DOI resolution at fetch time. A doi on a record is what the bytes are cited as, and it routes nothing. Resolving one through a repository API on every fetch would put a second host, which is not itself mirrorable, in front of every retrieval, and would break the mirror loop’s assumption that the locations it walks are independent sources. The archives are read when the registry is written instead: registry_draft() covers Zenodo, figshare and Dataverse, and emits plain locations.
  • Reading data. getaca returns a path and knows nothing about file formats. A processor can unpack an archive; nothing reads its contents.
  • Publishing. Uploading the data, minting the version, computing the checksum in the first place: all outside.
  • Semantic version solving. Version strings are labels, and the registry names the head rather than ranking them.
  • Workflow orchestration. Composing with targets is the answer, rather than growing a dependency graph.

The list is short on purpose. Each item is a place where a package could grow into something else, and the dependency footprint that makes getaca inexpensive to depend on is the thing that would go first.

packageDescription("getaca")$Imports
#> [1] "curl (>= 5.0.0), stats, tools, utils"

Where to go next