Make a version the one new evaluations use
Existing answers keep their judgment_version until recomputed.
Activating a version whose engine or definition differs from the active
one without force returns a shadow job in awaiting_confirm
(202). The job evaluates a random sample of 1,000 documents under the
new version, never producing answers, and its report compares old and
new. POST /jobs/{id}/confirm completes the switch, at any
point; cancelling the job leaves the active version as it was. With
force: true, or when nothing differs, the version is active at once
(200).
On a prefix path the sample is drawn across every namespace under the
prefix, and confirming switches them all. A namespace cannot activate a
version of a judgment it inherits (conflict): it follows the
template’s active version.
Activating a composite version always returns the shadow job,
even with force. Its sample is documents with outcomes for this
judgment, each scored on held-out labels against a fair
baseline with its own fitted cut-off (CompositeShadowReport).
Fewer than 50 labelled documents, or fewer than 10 of either answer,
fail the job with insufficient_labels. confirm activates the
version with the combiner fitted on all of them, whatever the
verdict.
Authorizations
An organization API key. Keys carry a role (read_write or
read_only) and may be restricted to a namespace prefix such as
acme/*, or to one namespace such as acme/prod/tenant_1. A prefix
matches on a / boundary: acme/prod/tenant_1* covers
acme/prod/tenant_1 and everything under acme/prod/tenant_1/,
never acme/prod/tenant_12.
Headers
One key per logical request, reused only on its retries. A key
belongs to one request: within your organization, the same method,
path, query and body. For 24 hours after a successful response, a
request with the key and the same body gets that response back
verbatim, with Idempotent-Replayed: true, and runs nothing. Only a
successful response is kept, so the retry of a request that failed
runs again. While the first request runs or its response is kept,
the key with a different request is idempotency_key_reused (422).
A request sent while one with its key is still running is
rate_limited with Retry-After: 1, without running: retry it to
get the first one's response. In the rare case the key can't be
checked, the request runs as if it had none.
1 - 255Path Parameters
A namespace name, or a template prefix ending in /*, with any
/ sent as %2F: acme%2Fprod%2Ftenant_123 or acme%2Fprod%2F*.
A namespace name, or a template prefix: a namespace path ending in
/*, such as acme/prod/*, which every namespace under acme/prod/
inherits judgments from. Up to 256 bytes. Never exactly
. or .., which a URL path can't carry.
1 - 256^(?!\.\.?$)[A-Za-z0-9._:/-]+(/\*)?$A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$Body
Response
The version is active.
A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$Null once the judgment is deactivated.
x >= 1Settings, not part of the definition. Changing them creates no version.
The judgment's named thresholds, a setting rather than part of a
version. Each is a number for a bool judgment (true when p is
at least it) or a score judgment (true when score is at least it),
and {value, gte} for a choice judgment (true when dist[value] is at
least gte). They are evaluated at read time against the raw fields,
never the calibrated ones. A threshold that does not fit the
judgment's type is invalid_request.
How the judgment's outcomes are read and derived, a setting rather than part of a version. See calibration.
Every version, oldest first. Versions are kept until the namespace is deleted.
Present when the judgment is inherited, the template prefix it comes from, such as acme/prod/*.
1 - 256^(?!\.\.?$)[A-Za-z0-9._:/-]+(/\*)?$Present when the judgment is inherited. The settings this namespace overrides; the others follow the template.
thresholds, freshness Where each value in freshness.fanout comes from, key for key. Present exactly when freshness.fanout is. A
where removed with null is the default, no filter; any other
null the judgment set, such as created_within: null, is the
judgment's.
Response only. How much of
freshness.fanout.rolling_limit the judgment's fan-outs have used:
the re-judgments counted in the rolling 30-day window. Present
exactly when freshness.fanout is. A fan-out that would take used
past the limit is deferred.
A setting of a fixed-option choice: when the
share of its answers over the last 7 days that are
none_of_the_above rises above this, the judgment gains the
taxonomy_drift warning and the events feed one
judgment.taxonomy_drift. Null (the default) turns it off. Nothing
changes on its own: run discover when you want proposals.
0 < x <= 1Conditions to act on, such as taxonomy_drift. Absent when there are none.
The judgments whose relations read this one's answers. While there are any, deleting or deactivating it, a policy other than on_change, or a version that is not plain or narrows applies_to below a reader's match is conflict naming them. Absent when there are none.
A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$Today's evaluations of the judgment in this namespace, UTC. The dashboard's per-judgment numbers. Before the day's
first evaluation only day, evaluations (0) and spend_usd (0)
are present.