Name alternatives to score
Adds alternatives to the active version. Each names one or
more of engine, context, question and criteria; the active
version fills the rest, and thresholds are never part of one.
Each is checked as a new version would be: an active engine, a
recipe that compiles, a question within limits. On a judgment with
features, a recipe must keep every relation a feature reads.
One that judges exactly like the active version is refused. At most
10 engine and named alternatives are scored a month.
A recipe that reads more than the active one (a new field or
relation, a larger last_n, a relation that selects by another
judgment’s answer) is scored on today’s related documents and
answers, not on what each labelled answer read, and its row says
as_of: now. A relation that selects by another judgment’s answer
needs that judgment active with answers on the related documents;
the quote prices that backfill on a line of its own.
Naming an engine this namespace does not use sends the labelled
contexts to that engine’s provider when the run is confirmed; the
quote names it in new_providers.
With run: true and confirm: true, starts or extends this
month’s run (202), against the quote of these alternatives; with
run: true alone, returns the quote. On the Team plan and above.
Authorizations
An organization API key. Keys carry a role (read_write or
read_only) and may be restricted to a namespace prefix such as
acme/*, or to one namespace such as acme/prod/tenant_1. A prefix
matches on a / boundary: acme/prod/tenant_1* covers
acme/prod/tenant_1 and everything under acme/prod/tenant_1/,
never acme/prod/tenant_12.
Headers
One key per logical request, reused only on its retries. A key
belongs to one request: within your organization, the same method,
path, query and body. For 24 hours after a successful response, a
request with the key and the same body gets that response back
verbatim, with Idempotent-Replayed: true, and runs nothing. Only a
successful response is kept, so the retry of a request that failed
runs again. While the first request runs or its response is kept,
the key with a different request is idempotency_key_reused (422).
A request sent while one with its key is still running is
rate_limited with Retry-After: 1, without running: retry it to
get the first one's response. In the rare case the key can't be
checked, the request runs as if it had none.
1 - 255Path Parameters
A namespace name, or a template prefix ending in /*, with any
/ sent as %2F: acme%2Fprod%2Ftenant_123 or acme%2Fprod%2F*.
A namespace name, or a template prefix: a namespace path ending in
/*, such as acme/prod/*, which every namespace under acme/prod/
inherits judgments from. Up to 256 bytes. Never exactly
. or .., which a URL path can't carry.
1 - 256^(?!\.\.?$)[A-Za-z0-9._:/-]+(/\*)?$A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$Body
1 - 10 elementsAlso start or extend this month's run; with confirm, it starts.
The outcomes a named run is fitted on. An outcome left out still counts as an answer's outcome, so it never makes that answer an implicit negative.
Response
The alternatives are named; the table, with the quote of what is unscored.
A version's alternatives: which engine, recipe or wording its outcomes prefer, what the labels can and cannot tell, and what scoring costs.
x >= 1The engine epoch of the calibration fit whose labelled answers the alternatives are scored on. Null before a fit.
not_run: nothing scored this month;reasonsays why.quoting: the quote is being priced; read again shortly.running: this month's run is scoring.ready:suggestionnames what the outcomes prefer.none: the run found nothing better beyond noise, or nothing cheaper;messagesays what was detectable.
not_run, quoting, running, ready, none Not run: awaiting_confirm (there is something to score and
automatic is off: confirm the quote), awaiting_run (the next
calibration run prices it), too_few_outcomes (no calibration fit
yet), composite, inherited (the template has the table),
budget_paused (the run waits for budget). None:
nothing_better (nothing beat the version beyond noise; message
says the smallest gain the labels could show),
accuracy_would_drop. A narrowing run's own reasons are
recipe tuning's. Null when ready, running or quoting.
awaiting_confirm, awaiting_run, too_few_outcomes, composite, inherited, budget_paused, nothing_better, accuracy_would_drop, superseded The status in a sentence.
Whether the run starts by itself after a fit, at most monthly (alternatives.automatic on the judgment). Off by default.
Engines the judgment's setting skips as automatic alternatives.
The labelled answers the alternatives are scored on.
Engine and named alternatives scored this month. Every better and moves_ranking holds after correcting for all of them.
x >= 0What scoring the unscored engine and named alternatives would bill, priced before anything runs, and the smallest gain the labels could show. Each answered call is billed as judging: its size class times its engine's weight, at the graduated prices of the tiers your organization is in now.
The active version on the same labels, scored the same way. Null before a run.
Engine and named alternatives first (in_k: true), then recipe tuning's narrowings.
The alternative the outcomes prefer, by id, and why. Nothing is applied.
The suggestion in a sentence, with the correction it holds after.
RFC 3339, UTC.
When the automatic run may next start. Null while automatic is off.
Below Team, why scoring engine and named alternatives, and the narrowings' recipes, are withheld.