Recommend a threshold for a precision or recall target
The threshold on the raw p (bool) or raw dist[option] (choice) that
meets the target on the active version’s outcomes. A document
counts as predicted positive when the raw value is at least the
threshold, and as positive when its outcome is true (bool) or equals
option (choice).
- For a precision target, the lowest threshold whose precision meets it, which keeps the most recall.
- For a recall target, the highest threshold whose recall meets it, which keeps the most precision.
For a choice with options.from the threshold is
on raw escape_p, the unmatched queue, and option is
none_of_the_above or left out. A document whose escape_p is
below the threshold has its pick taken; at or above it, it waits in
the queue. Precision is the share of taken picks that equal the
outcome, and recall the share of outcomes naming a match whose pick
was taken and right, so a precision target takes the highest
threshold that meets it, and a recall target the lowest. Apply it as
{"unmatched": {"value": "none_of_the_above", "gte": threshold}}.
The recommendation rests on the outcomes of the current engine epoch:
at least 100, with at least 20 positive and 20 negative (for a choice,
20 of option and 20 of the other options). When that epoch has too
few, it uses the previous epoch’s and says so with
from_previous_epoch, as calibration does. When neither has enough,
insufficient_data says why and what to post. Nothing
changes until the threshold is applied with PATCH. Score judgments have no recommendation (invalid_request).
The recommender is part of the learning loop, on the Team plan and
above: below it, plan_required. The calibration report and
calibrated answers are on every plan.
Authorizations
An organization API key. Keys carry a role (read_write or
read_only) and may be restricted to a namespace prefix such as
acme/*, or to one namespace such as acme/prod/tenant_1. A prefix
matches on a / boundary: acme/prod/tenant_1* covers
acme/prod/tenant_1 and everything under acme/prod/tenant_1/,
never acme/prod/tenant_12.
Path Parameters
A namespace name, or a template prefix ending in /*, with any
/ sent as %2F: acme%2Fprod%2Ftenant_123 or acme%2Fprod%2F*.
A namespace name, or a template prefix: a namespace path ending in
/*, such as acme/prod/*, which every namespace under acme/prod/
inherits judgments from. Up to 256 bytes. Never exactly
. or .., which a URL path can't carry.
1 - 256^(?!\.\.?$)[A-Za-z0-9._:/-]+(/\*)?$A judgment, attribute or threshold name. Names are path segments in field references, so they never contain ..
1 - 128^[A-Za-z0-9_-]+$Query Parameters
The metric and the value to meet, such as precision:0.9 or recall:0.8.
A metric and the value to meet, such as precision:0.9 or recall:0.8.
^(precision|recall):(0?\.[0-9]*[1-9][0-9]*|1|1\.0+)$Required for a choice judgment, and refused for a bool one. The option the threshold is for, such as fraud.
1Response
A recommendation, an unreachable target, or too few outcomes; status says which.
- Option 1
- Option 2
- Option 3
Apply threshold with the judgment PATCH: {"escalate": threshold} for a bool judgment, {"fraud": {"value": option, "gte": threshold}} for a choice.
The threshold on the raw value, one of the curve's points.
0 <= x <= 1Precision on these outcomes at the threshold.
0 <= x <= 1Recall on these outcomes at the threshold.
0 <= x <= 1The 95% interval of the targeted metric at the threshold.
The number of outcomes the recommendation rests on.
x >= 100True when the current engine epoch has too few outcomes (fewer than 100, or fewer than 20 of either class) and the previous epoch's were used.
Precision and recall at each threshold from 0.00 to 1.00 in steps of 0.01, lowest first: 101 points.
101