Make an alternative a version, and see its shadow report
Creates the alternative’s definition as a new version, as
POST /judgments would, and starts its shadow job, as activate
without force would (202). The shadow report carries
alternative: the outcomes’ verdict and how many alternatives it
was corrected for when the version was created, beside the
backfill’s price in recompute and, on another engine,
calibration_replay. Confirming the job is the one step that
activates it.
The new version’s calibration is fitted as any new version’s is,
and its calibration report’s alternative_selection shows the numbers it was
chosen on beside the same numbers on outcomes observed since.
On the Team plan and above.
Authorizations
An organization API key. Keys carry a role (read_write or
read_only) and may be restricted to a namespace prefix such as
acme/*, or to one namespace such as acme/prod/tenant_1. A prefix
matches on a / boundary: acme/prod/tenant_1* covers
acme/prod/tenant_1 and everything under acme/prod/tenant_1/,
never acme/prod/tenant_12.
Headers
One key per logical request, reused only on its retries. A key
belongs to one request: within your organization, the same method,
path, query and body. For 24 hours after a successful response, a
request with the key and the same body gets that response back
verbatim, with Idempotent-Replayed: true, and runs nothing. Only a
successful response is kept, so the retry of a request that failed
runs again. While the first request runs or its response is kept,
the key with a different request is idempotency_key_reused (422).
A request sent while one with its key is still running is
rate_limited with Retry-After: 1, without running: retry it to
get the first one's response. In the rare case the key can't be
checked, the request runs as if it had none.
1 - 255Path Parameters
A namespace name, or a template prefix ending in /*, with any
/ sent as %2F: acme%2Fprod%2Ftenant_123 or acme%2Fprod%2F*.
A namespace name, or a template prefix: a namespace path ending in
/*, such as acme/prod/*, which every namespace under acme/prod/
inherits judgments from. Up to 256 bytes. Never exactly
. or .., which a URL path can't carry.
1 - 256^(?!\.\.?$)[A-Za-z0-9._:/-]+(/\*)?$A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$An alternative's id, as getAlternatives lists it.
An alternative's id, the same for the same definition in every run.
^alt_[0-9a-f]{16}$"alt_3f9a2c7e1b04d6a8"
Response
The version is created and its shadow job awaits confirmation.
A job. A shadow job comes from activating a differing
version without force. It starts in awaiting_confirm and
runs its sample while it waits; report fills in when the sample is
done. confirm, accepted once report is filled in, activates
version, starts a backfill of the answers from versions that ask
differently (backfill_job_id), and the job ends done; cancel
leaves the active version unchanged. On a template prefix, namespace
is the prefix. A reference_index job builds the index
an entity judgment's relations need; it starts running, and
attribute names the attribute it indexes. An evaluation_export
job starts running, counts evaluations written in
progress.documents_done, never spends, and has export.
A resync job re-renders the readers of a changed threshold (resync) and re-judges the documents whose rendering flipped; a group_build job computes a new group's or aggregate's
first aggregates with no fan-out (group); a discover job fills
in proposal; a simulation job fills in simulation. Each starts running; discover and simulation never
spend. Fan-out is not a job: it runs within the judgment's rolling
limit and is deferred past it.
A judgment_create, judgment_update or backfill_start job
finishes a create, PATCH or backfill request that needed more than
20 s (that request's 202). It starts running and ends done when
the request succeeded or failed when it did not; either way
result is the response the request would have returned, and
error is the error's code and message when it failed. It never
spends, and it cannot be paused or cancelled.
backfill, shadow, periodic, namespace_delete, reference_index, evaluation_export, resync, group_build, discover, simulation, judgment_create, judgment_update, backfill_start estimating, awaiting_confirm, running, paused, done, failed, cancelled A namespace name, or a template prefix: a namespace path ending in
/*, such as acme/prod/*, which every namespace under acme/prod/
inherits judgments from. Up to 256 bytes. Never exactly
. or .., which a URL path can't carry.
1 - 256^(?!\.\.?$)[A-Za-z0-9._:/-]+(/\*)?$Judging billed by this job so far, in US dollars.
x >= 0RFC 3339, UTC.
RFC 3339, UTC.
A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$RFC 3339, UTC.
Only on reference_index jobs, as attributes.<name>. The attribute whose reference index the job builds. Null for other jobs.
^attributes\.[A-Za-z0-9_-]+$Only on resync jobs.
Only on group_build jobs, the group whose aggregates it computes.
1 - 128^[A-Za-z0-9_-]+$Only on discover jobs; null until done.
Only on simulation jobs; null until done.
Shadow jobs only. The version being activated.
x >= 1Shadow jobs only. Null while the sample runs; progress counts the
sampled documents. A composite version's job has a
CompositeShadowReport, and when it has too few labels it
fails with an error that starts with insufficient_labels.
- Option 1
- Option 2
Shadow jobs only. The backfill job that confirming started once
the version was active, which judges again each document whose
answer is from a version that asks differently. Its estimate is
the report's recompute, which the confirm agreed to. Null until
the job is done, and also then when there was nothing to judge
again (the judgment has no other version that asks differently,
such as a composite's first version, or the recompute counts no
documents), or when the recompute did not fit what is left
of the namespace's budget this billing period or the
organization's balance. Then the switch still happened: start a backfill yourself,
which shows its estimate first.
evaluation_export jobs only: the range, the evaluations written, and once done the download links.
Only on judgment_create, judgment_update and backfill_start jobs. Null until the job is done or failed; then the response the request would have returned.
Only in the response of the activation that started a shadow job, such as images_unread. Absent when there are none, and on GET /jobs/{id}.