When a new fit replaces the current one

​

When fits run

• The first run starts within a few minutes of the first outcome you post, or of the judgment’s first outcome rules. A run makes a fit only for an epoch that meets the minimums, so the first run usually makes none.
• Until there is a fit, Brussle runs again every hour (for the current epoch, and for a new epoch with no fit yet), so the first fit comes within the hour after your outcomes meet the minimums.
• Once there is a fit, Brussle makes a new fit each night on all the outcomes so far, rule outcomes included. The new fit replaces the current fit only under the rule.
• When Brussle judges answers again to correct them, the next run starts within a few minutes after the correction ends (see when answers are corrected).

​

Fit now

• You can ask once an hour for each judgment, and named runs share this limit. Within that hour the call returns rate_limited (429), and Retry-After gives the seconds until Brussle accepts the next call.
• If a run is already waiting or running, that run serves the call.
• Fit now runs on every plan, and it is never billed.

​

The rule

1. It is clearly better on the outcomes that arrived since the current fit was made. The current fit never saw them, and the challenger is scored on each one as a fit that never saw it, so they are a fair test of both. The challenger must beat the current fit on them by more than chance would explain.
2. It moves the number by at least a point: 0.01 on average over all the judgment’s outcomes. For a choice or score, the move is the probability that moves between the options.

• An outcome arrives when you post it, not when it was observed. An implicit negative arrives when its window closes. After the engine changes, a replayed outcome arrives when it is replayed. An outcome whose answer was judged again with changed inputs counts as new too.
• The current fit is withdrawn when the engine’s raw numbers beat it by the same margin on the outcomes since. Answers then read the night’s fit, or the raw numbers if that fit did not beat them. The run says withdrawn: fallback_better.
• A fit held for its labels withdraws the current one. When the night’s fit is held because the labels cannot support a correction, truncated_sample or labels_read_like_the_outcome on the report, the current fit was made on those same labels and is withdrawn without the test: answers read the engine’s numbers, and the run says withdrawn: labels_held. Nothing waits for new outcomes; the next run after the labels cover the live answers, or stop reading like the outcome, fits again. A pinned fit stays pinned.
• A new start takes the night’s fit at once: the first fit, a new engine epoch, a new version, and a change from fitting on all outcomes to fitting on labelling-queue labels alone, or back.
• Below 100 outcomes (while the current fit or the night’s rests on fewer than 100), each night’s fit replaces the current one under the margin below 100 rather than this test.
• When a run cannot fit again, because outcomes were deleted and the epoch fell below the minimums, the current fit stays.
• It applies to every judgment: to calibrated on a bool, a choice and a score, to p of a judgment with features, and to a template tenant’s own fit and its own weights, on that tenant’s own outcomes. The exception is a choice among candidates with features, where each run’s fit replaces the current one, as a first fit does.
• On a steady process, the rule rarely changes the fit by chance. A real change in your outcomes (a new base rate, say) shows in the outcomes since, and the new fit replaces the current one once it is clearly better.

​

Read the current fit and the challenger

• current: the fit answers read now. Its fit is its id, or null when it did not beat the raw numbers, and run is the run that holds it. pinned is true for a pinned fit, and pin_suspended is older_preprocessing when the pin is suspended. fitted_at is when the run that made the fit ran. A fit kept since an earlier run keeps that date, so it can be weeks older than the newest run. since measures the fit against the raw numbers on the outcomes that arrived after it was made: outcomes, fallback_log_loss (the raw numbers’), log_loss (the current fit’s) and the standard_error of the difference. since is null while those outcomes hold fewer than 20 of each kind, and on the night the fit was made. current is null while the epoch has no fit of its own.
• challenger: the newest run’s fit when it did not replace the current one, and null when it did or when there was no current fit. It has its own fit, run, fitted_at, outcomes and held_out, and versus_current, which sets the challenger against the current fit on the outcomes since. gain is the current fit’s log loss minus the challenger’s, so a positive gain favours the challenger. shift is how far the challenger would move the number on average over every outcome. gain and standard_error are null with fewer than 20 of each kind since. On a choice or score with features, moved_picks is the share of outcomes whose most probable option would change.

​

Why a new fit was held

• too_few_outcomes_since: fewer than 20 of each kind arrived since the current fit was made. Post more outcomes, or wait for them.
• not_clearly_better: the new fit did not beat the current one clearly on the outcomes since (versus_current.gain and standard_error give the margin). The difference is within noise.
• too_small_a_move: the new fit is better but would move the number by less than 0.01 on average (versus_current.shift). On a judgment with features each move re-judges the judgments that read p, billed, so a move this small is not worth it.
• pinned: a pin holds the fit.

​

Promote the newest fit

const answer = await ns.judgments.fitCalibration("material_change", { promote: true });
if (!("queued_at" in answer)) {
  // An estimate: other judgments read this one. Read `answer.downstream`, then confirm.
  await ns.judgments.fitCalibration("material_change", { promote: true, confirm: true });
}

answer = ns.judgments.fit_calibration("material_change", promote=True)
if "queued_at" not in answer:
    # An estimate: other judgments read this one. Read answer["downstream"], then confirm.
    ns.judgments.fit_calibration("material_change", promote=True, confirm=True)

• The run makes its own new fit and applies it. It fits all the outcomes so far, as a nightly run does, and that fit becomes current for each version and epoch that is not pinned, tenants included. A promote never moves a pin.
• With readers, confirm first. On a judgment with features (a bool, a choice or a score) that other judgments read, a new current fit judges their documents again, billed. Without confirm, the call returns the downstream estimate (200), made against last night’s fits, and queues nothing. With confirm: true, Brussle queues the run, and the answer carries the estimate too.
• A confirmed promote is not held by the hourly limit. You accepted its cost, so Brussle queues it even within an hour of Fit now. Without readers, Brussle queues a promote at once, and it counts as Fit now’s request for the hour.
• Templates. On a namespace that inherits the judgment, a promote is invalid_request; promote on the template. Fit now without a body still queues the template’s run.

​

Pin a fit

{"calibration": {"pins": [
  {"run": "01JA2J0P2R4T6W8Y0A2C4E6G9B", "version": 3, "engine": "jev", "engine_version": "current+2026-09-24.1", "fit": "fit_9f3a2c7e1b04d6a8"}
]}}

const pins = [{ run: "01JA2J0P2R4T6W8Y0A2C4E6G9B", version: 3, engine: "jev", engine_version: "current+2026-09-24.1", fit: "fit_9f3a2c7e1b04d6a8" }];
await ns.judgments.update("material_change", { calibration: { pins } });

pins = [{"run": "01JA2J0P2R4T6W8Y0A2C4E6G9B", "version": 3, "engine": "jev", "engine_version": "current+2026-09-24.1", "fit": "fit_9f3a2c7e1b04d6a8"}]
ns.judgments.update("material_change", calibration={"pins": pins})

• Send calibration alone in its PATCH /namespaces/{ns}/judgments/{name}, without other settings, or the request is invalid_request.
• The list replaces all the pins. Include the pins you want to keep. A pin sent again from the same run stays as it is, and two pins for one version and epoch are invalid_request.
• The pin writes a run (cause: pin) that holds the pinned fit, and every later run holds it too. The response has the judgment, with calibration: {"mode": "pinned", "pins": [...]}, and each pin has pinned_at. mode is auto when no pin applies.
• Answers of that version and epoch read the pinned fit, also when a later fit would do better. Every other version and epoch stays auto. Each run still makes a fit and lists it as the challenger, and when that fit clearly beats the pinned fit on the outcomes since, the report warns pinned_fit_beaten.
• A pin is a setting, like thresholds. It makes no version, and a new version starts in auto.
• On any judgment without features, a pin changes only calibrated. Readers use the raw numbers, so nothing is judged again or billed, and there is nothing to confirm.

const result = await ns.judgments.update("material_change", { calibration: { pins } });
if ("downstream" in result && !("judgment" in result)) {
  // Nothing changed. Show each reader's cost, then confirm.
  console.log(result.downstream);
  await ns.judgments.update("material_change", { calibration: { pins }, confirm: true });
}

result = ns.judgments.update("material_change", calibration={"pins": pins})
if "downstream" in result and "judgment" not in result:
    # Nothing changed. Show each reader's cost, then confirm.
    print(result["downstream"])
    ns.judgments.update("material_change", calibration={"pins": pins}, confirm=True)

• the run is not in the judgment’s list of fits
• the run has no fit for that version and epoch (for example, because it did not beat the raw numbers)
• fit is not that run’s fit
• the fit uses features and was made before Brussle last changed how it keeps features
• the namespace inherits the judgment from a template (set the pin on the template, as per-customer corrections shows)

​

A pin and a new epoch

​

A suspended pin

​

Lift a pin

await ns.judgments.update("material_change", { calibration: { pins: [] } });

ns.judgments.update("material_change", calibration={"pins": []})

• The next run comes at once. Brussle brings it forward, usually within a few minutes. It does not count toward Fit now’s limit, and the list of fits shows it as cause: nightly.
• Until that run ends, answers read the pinned fit and still show "pinned": true. GET on the judgment already shows mode: auto.
• A lift keeps the number until a clearly better fit arrives. The formerly pinned fit stays current, now in auto, and that run’s fit replaces it only when it is clearly better on the outcomes since the pinned fit was made (usually many by then). To take tonight’s fit anyway, promote. On a judgment with features, when the fit its answers read changes, Brussle re-judges the readers and bills for them, with no confirm.

​

Events when the fit changes

• judgment.fit_applied reports how many documents crossed each of the judgment’s thresholds. When the numbers go back to the engine’s, the event is judgment.fit_withdrawn. Both carry fit, run and pinned.
• The event gives each reader’s re-judges: engine_judgments, which are billed, and reuses (re-judges where only a feature the reader keeps moved), which use their engine answer again at no charge.
• The counts come from those re-judges, so the event arrives after they have started.