The run from targets and series, the combiner over its candidates, and fitting across a set of grains on its own.
timesift()
timesift(
targets,
series=None,
*,
y,
x=None,
id=None,
time=None,
target_time=None,
static=None,
models=None,
sift=None,
ensemble=True,
resampling=None,
response: str = 'presence_absence',
metric=None,
control=None,
keep_fits: bool = False,
verbose: bool = True,
)Fit every learner across every representation, on one fold map, and combine them.
targets is one row per thing to predict and
series is the long, time-stamped record belonging to it;
both are mappings of column name to array, which a data frame satisfies.
y, x and static are selections
over their own table: a name, a list of names, a glob such as
"sp_*", or a function of a name.
Columns of targets that are neither the response nor the
identifier nor the anchor are ignored unless static names
them: a predictor is never picked up because it happened to be in the
table.
Timesift
Timesift(
candidates,
scores,
oof,
representations,
sift,
stack,
weights,
models,
folds,
cells,
y,
metric,
scorer,
response,
spec,
fits,
control,
)A fitted sift: every candidate’s out-of-fold predictions, their scores, and the combiner.
candidates and scores are tables held as
columns; oof, models and fits are
keyed by candidate name. What reads them – the summary, the weights, the
occlusion profile – reads them and never a model, so a number reported
here was read where the score was.
Attributes:
-
candidates- dict -
scores- dict -
oof- dict -
representations- dict -
sift- Sift -
stack- object -
weights- object -
models- dict -
folds- Folds -
cells- object -
y- Response -
metric- str -
scorer- object -
response- str -
spec- TimesiftSpec -
fits- dict -
control- object
TimesiftSpec
How a fit was asked for: the columns each table plays, and the calendar they are read in.
Attributes:
-
y- tuple[str, …] -
x- tuple[str, …] -
id- str | None -
time- str | None -
target_time- str | None -
static- tuple[str, …] -
response- str -
metric- object
target_labels()
What names the rows of every array in one fit.
The unit identifier names them where a unit carries one target. Where
target_time lets a unit carry several, no identifier tells
them apart, so the row’s own position does, which is what
timesift.representation.lookback_matrix already names its
targets by.
select_columns()
The columns a selection names, in the order the selection names them.
An explicit list is taken in the caller’s order, because naming the response columns in an order is a decision; a glob and a predicate are taken in the table’s own order, because matching is not an ordering.
exclude names columns that exist but cannot be selected
here, so naming one is refused with the reason rather than reported as a
column that does not exist.
candidate_table()
One row per candidate: its level, how many responses it was best on, and how it covered them.
A candidate whose learner and representation could not be paired carries no level, and is listed under the ones that do.
ensemble()
Ask for an ensemble of the candidates a fit produced.
stack fits non-negative weights summing to one on the
out-of-fold predictions, mean and median
combine without fitting, and weighted uses each candidate’s
own mean score rescaled to sum to one. scope is which
candidates are eligible: every one of them, only the several learners
sharing the best candidate’s representation, or only its learner across
the representations. metric names the metric the ensemble
is reported in, or None for the fit’s own, and
response is the registered head whose loss the weights
minimise, or None for the head the run was fitted under.
Naming a head the run does not fit toward is an error rather than an
override, and ensemble_fit called on its own reads
None as "presence_absence".
ensemble_fit()
Fit the combiner on the out-of-fold predictions and nothing else.
oof is one [target, response] matrix per
candidate, in the response’s own row order. Only the cells the mask
admits are read, so every candidate is weighted on the same cells its
score was read on.
ensemble_weights()
The weight the combiner gave each of its members, or nothing where a run combined none.
EnsembleSpec
How the candidates are to be combined, which of them are eligible, and under which head.
Attributes:
-
method- str -
scope- str -
metric- object -
response- str | None
Stack
A fitted combiner: what it does and what weight it gave each of its members.
Attributes:
-
method- str -
weights- dict -
members- tuple[str, …]
grain_ladder()
grain_ladder(
x,
y,
learners,
folds=None,
response: str = 'presence_absence',
metric=None,
control=None,
keep_fits: bool = False,
interval: str = 'variables',
repeats: int = 1,
seed: int = 1,
verbose: bool = True,
)Cross-validate every learner at every grain, on one fold map and one mask of cells.
Every arm sees identical splits and is restricted to identical cells,
so the arms’ means share a denominator and any two of them can be
compared with paired_contrast.
folds left at None builds one with the
defaults of fold_map. Where both languages must see the
same splits, build it once and read it in the other with
read_folds.
control is the train_control every neural
learner of the ladder trains under; a learner carrying settings of its
own overrides it on the ones it names.
interval="nested_cv" refits every arm inside every outer
training set of every repetition, which is what
paired_contrast(interval="nested_cv") reads an interval for
the difference in risk off. Two tables whose contrast is to be read take
the same folds, repeats and
seed.
fit_learner()
fit_learner(
learner,
x: TimesiftMatrix,
y,
response: str = 'presence_absence',
control=None,
group=None,
**kwargs,
)Fit one learner at one grain, under one registered response head.
A fit that declares a head argument is
handed the registered head, whose loss and
activation say what it is fitting toward; the learners that
ship read both from there and hold no response of their own. A
fit that declares a control is handed the
run’s training settings the same way.
select_grain()
select_grain(
x,
y,
learners,
folds=None,
inner=5,
rule: str = 'argmax',
threshold: str | None = None,
interval: str = 'variables',
repeats: int = 1,
response: str = 'presence_absence',
metric=None,
compare: Ladder | None = None,
control=None,
seed: int = 1,
verbose: bool = True,
)Choose the grain inside each outer fold’s training units, then score the whole procedure.
Within each outer fold the training units are split again, every candidate is fitted on part of them and scored on the rest, the best is refitted on the whole outer training set, and the outer test fold is predicted once. The estimate that comes back is therefore of the procedure including its choice of grain, which is what an ecologist applying it to a new site would run.
What the estimate is of: the expected held-out score of the whole pipeline, selection included, on units drawn as these were. What it is not: the score of the winning grain. That is higher, by the amount selection buys itself, and the difference between the two is the quantity this function exists to keep out of a reported number.
The cost is the ladder’s, multiplied by the number of inner folds:
v_outer * (v_inner * candidates + 1) fits.
control is the train_control every neural
learner trains under, in the inner search and in the refit alike; a
learner carrying settings of its own overrides it on the ones it
names.
Inside each outer fold every candidate carries an inner score, the
mean over variables of its per-variable mean over the inner folds, and a
standard error, the standard deviation over the inner folds of the
fold’s own score divided by the square root of their number.
rule chooses among them. "argmax" takes the
highest score, and on an exact tie the candidate declared first.
"coarsest_adequate" is the one-standard-error rule
(Breiman, Friedman, Olshen and Stone 1984; Hastie, Tibshirani and
Friedman 2009, section 7.10) with coarseness in place of complexity:
every candidate scoring at least the highest minus its standard error is
adequate, and the one with the fewest bins wins, then the fewest
channels, then the higher score, then the one declared first. A standard
error that cannot be computed is taken as zero.
threshold names a rule of
decision_threshold ("youden", the cut that
maximises TSS, "kappa" or "prevalence"). With
it set, each outer fold learns one cut per variable on the inner
out-of-fold predictions of the candidate it selected, which cover the
outer training units and nothing else, freezes it, and reads the outer
test fold’s predictions at it with tss. The estimate then
carries a row tss_inner_cut, thresholds the
cut of every outer fold and variable, and cut_scores the
per-cell rows.
The estimate carries the interval across the response variables,
which is the spread between the variables of this dataset rather than an
interval for what the procedure would score on a new sample.
interval="nested_cv" adds one that is, by the nested
cross-validation of Bates, Hastie and Tibshirani (2024): each outer
training set is cross-validated again over the remaining folds of the
same map, over repeats fold maps, which gives the mean
squared error of a cross-validation estimate; final then
holds the procedure fitted on every unit, whose risk the interval is
for. One repetition costs one fit of the procedure per unordered pair of
outer folds.
Ladder
Ladder(
grain,
learner,
variable,
fold,
score,
scorable,
predictions,
cells,
folds,
metric,
scorer,
response,
fits,
ncv,
)One score per (grain, learner, variable, fold) cell, and
what produced it.
Attributes:
-
grain- np.ndarray -
learner- np.ndarray -
variable- np.ndarray -
fold- np.ndarray -
score- np.ndarray -
scorable- np.ndarray -
predictions- dict -
cells- object -
folds- Folds -
metric- str -
scorer- object -
response- str -
fits- dict -
ncv- dict | None
Fit
A fitted learner, the variables it was fitted on, and the bins and channels of the representation it was made on.
A representation asked to predict is checked against those once, before any learner sees it: a calendar grain’s bins are named by their starts, so a record from another period is refused by the first bin that differs rather than read by position.
Attributes:
-
learner- Learner -
model- object -
variables- tuple[str, …] -
response- str -
bins- tuple[str, …] -
channels- tuple[str, …]
Selection
Selection(
selected,
estimate,
contrast,
candidates,
scores,
inner,
metric,
response,
rule,
threshold,
thresholds,
cut_scores,
interval,
nested_cv,
final,
)What a nested selection chose, what it scores, and what it was searched over.
Attributes:
-
selected- list[dict] -
estimate- list[dict] -
contrast- list[dict] | None -
candidates- list[dict] -
scores- Ladder -
inner- list[dict] -
metric- str -
response- str -
rule- str -
threshold- str | None -
thresholds- list[dict] | None -
cut_scores- Ladder | None -
interval- str -
nested_cv- list[dict] | None -
final- dict | None