> ## Documentation Index > Fetch the complete documentation index at: https://docs.brussle.com/llms.txt > Use this file to discover all available pages before exploring further. # The correction > Brussle fits a correction to each judgment's number on the outcomes you record, applies it only when it beats the engine's own number on held-out outcomes, and gives every corrected number a 95% interval to automate on. The engine says 0.8, and a column stores 0.8, but nothing says what 0.8 means on your data. Once you record what happened, Brussle fits a **correction** to the number from your outcomes. Every answer then carries the corrected number beside the engine's, with a 95% interval you can automate on and the id of the fit that made it. Brussle applies the correction only when it beats the engine's own numbers on outcomes it did not see. The engine's weights never change. Without [features](/calibration/features), the correction changes the number and never the order of your documents. You decide what to ask and what an answer means. Your outcomes measure whether a judgment answers your question on your data, which Brussle does not promise. ## How it works 1. You [record what happened](/calibration/outcomes) to judged documents: labelled examples, results that come [later](/calibration/horizons), or outcomes that rules make from your own writes. 2. Brussle keeps each outcome beside the exact answer it measures, with that answer's document revision, judgment version and engine version. 3. Brussle fits a correction each night for each judgment version and engine [epoch](/calibration/engine-changes), and tests it on held-out outcomes (outcomes that the fit did not use). A fit is one correction, with an id you can [list, compare and pin](/calibration/fits). 4. A new fit replaces the one your answers read only when it is [clearly better](/calibration/promotion). 5. Brussle applies the fit when an answer is read, so a new fit updates every answer without judging anything again. The [calibration report](/calibration/report) shows the numbers before and after, on your held-out outcomes. Calibration from the outcomes you post is on every plan, as is the report, and fits are never billed. ## What changes in an answer When a fit uses enough outcomes and [beats the raw numbers](#only-when-it-beats-the-raw-numbers), each answer gets a `calibrated` object beside the engine's numbers: ```json theme={"theme":{"light":"css-variables","dark":"css-variables"}} { "invoice_settlement": { "type": "bool", "p": 0.91, "calibrated": {"p": 0.84, "interval": {"lower": 0.8, "upper": 0.87}, "stage": "full", "outcomes": 1432, "fit": "fit_9f3a2c7e1b04d6a8", "run": "01JA2J0P2R4T6W8Y0A2C4E6G9B", "pinned": false, "from_previous_epoch": false, "extrapolated": false}, "thresholds": {"review": true}, "freshness": "fresh", "engine_version": "current+2026-09-24.1" } } ``` * **The engine's `p`, `dist` and `score` never change.** `calibrated` sits beside them, with the fields of the answer's type: `p` for a `bool`; `value`, `dist` and `escape_p` for a `choice`; `score` and `dist` for a `score`. * **`interval`** is the 95% interval of the corrected number (see [how sure a corrected number is](#how-sure-a-corrected-number-is)). * **`stage`** is `early` or `full` (see [when you get a correction](#when-you-get-a-correction)). **`outcomes`** is how many outcomes the fit uses. * **`fit`, `run` and `pinned`** name the [fit](/calibration/fits) that made the number, the run the answer was read with, and whether the fit is [pinned](/calibration/promotion#pin-a-fit). * **`prior`**, on an `early` fit, says how much of the correction is your own outcomes' (see [below 100 outcomes](#below-100-outcomes)). * **`from_previous_epoch: true`** means the engine changed and the answer reads the previous epoch's fit until the new epoch has its own (see [when the engine changes](/calibration/engine-changes)). * **`extrapolated: true`** means the raw value is outside the range the fit was made on (below the lowest or above the highest raw value among its outcomes), so the corrected number is a guess from the edge of what the fit has seen. For a `choice` or `score`, the raw value is the probability of the most probable option or level. Post outcomes for documents like this one. * **`calibrated` is absent while there is no fit.** It is not `null`. ### Which number your thresholds and queries read Every reader (thresholds, filters, ranking, subscriptions, relations and groups) uses the number that the answer shows. * **A judgment without features:** readers use the engine's numbers, never `calibrated`, so a new fit never moves a document across a threshold without your action. To act on corrected numbers, pick a threshold on the engine's `p` with the [threshold recommender](/calibration/thresholds), which measures it against your outcomes, or read `calibrated` in your own code. * **A judgment with [features](/calibration/features):** the answer's own numbers are the corrected ones (`p`; or `dist`, `escape_p`, `value` and `score`), and `raw` holds the engine's. Readers act on the corrected numbers, so a new fit can move documents across a threshold, and on a `choice` it can change which option or candidate is picked. * **A [composite](/guides/composite-judgments):** it has no `calibrated` object. Its `p` comes from weights that Brussle fits on your labels in each calibration run, so a new set of weights moves `p`, and with it thresholds, filters, ranking and subscriptions. `GET` shows no `fit_changed` warning for a composite. The result is usually close to calibrated, but not always, so check the reliability curve in its report. ## When you get a correction Brussle makes a separate fit for each judgment version and each engine epoch, since a different question, or a different engine behaviour, needs a different correction. | Outcomes for that version and epoch | What happens | | - | - | | Fewer than 50, or too few of one kind | No correction. Answers have only the engine's numbers. | | 50 to 999, with at least 15 of each kind | `stage: early`. The correction is careful on purpose: it starts from the engine's own numbers and leans on your outcomes more as they grow, so expect it to move more between fits. | | 1,000 or more | `stage: full`. The correction follows your outcomes closely, where the engine is too sure and where it is not sure enough. | `stage` is a summary of how settled the fit is. Whatever the stage, automate on the [interval](#automate-on-the-bound), never on the number alone. ### At least 15 of each kind 50 outcomes are not enough on their own. An epoch also needs outcomes on both sides: * **`bool`:** at least 15 `true` and at least 15 `false`. * **`choice` and `score`:** at least two different values, with 15 outcomes each. The fit does not need every option, because most options are rare, but it needs to see the engine's top answer both right and wrong. A fit on one kind of outcome sees only that kind. If every outcome is `true`, the "best" correction says that everything is probable, which is wrong for every document where the event did not happen. Until an epoch has both kinds it gets no correction, and the report says why: ```json theme={"theme":{"light":"css-variables","dark":"css-variables"}} {"reason": "too_few_negatives", "message": "Only positive outcomes so far: post outcomes for documents where it did not happen."} ``` See [post what did not happen too](/calibration/outcomes#post-what-did-not-happen-too). The [threshold recommender](/calibration/thresholds), [recipe tuning](/calibration/recipe-tuning) and [alternatives](/calibration/alternatives) act on a correction, so they need more: 100 outcomes, with 20 of each kind. A judgment's [features](/calibration/features) need 100 outcomes too, with the fit's 15 of each kind. The first fit comes within the hour after your outcomes meet the minimums, or at once with **Fit now** (see [when fits run](/calibration/promotion#when-fits-run)). ## Below 100 outcomes A judgment gets a correction from 50 outcomes. Three rules keep it safe while your outcomes are few. * **It starts from the engine's own numbers.** The correction is part your outcomes and part the engine's numbers taken at their word. Your share is `prior.weight`, on each corrected answer and in the report. It is half at 50 outcomes and two thirds at 100. It reaches 86% at 300 and grows with each outcome, so the correction moves away from the engine's numbers only as fast as your outcomes support. From 1,000 outcomes the correction is your outcomes' alone, and `prior` is absent. `prior.kind` is `identity` (the engine's own numbers), or `pool` on a template's tenant, which leans on its template's pool instead (see [per-customer corrections](/calibration/tenants)). * **It must win clearly.** Below 100 outcomes, the correction must beat the raw numbers on outcomes it did not see, by more than chance would explain. When it wins by less, the report's `not_fitted` is `within_noise`, with `gain_standard_errors`. Calibration is working and not yet sure, and answers stay raw until it is. From 100 outcomes, any win applies. That win is over the engine's raw numbers, so it decides whether a fit applies at all, while a new fit replaces a current one only when it is clearly better than that fit (see [the rule](/calibration/promotion#the-rule)). * **A judgment with features waits for 100.** Its thresholds, filters and readers act on its `p`. Below 100 outcomes Brussle reports its fit and never applies it: `p` stays the engine's, and `not_fitted` is `readers_need_full`. * **The labels must cover the live answers, and must not read like the outcome.** A fit made on labels from one part of the range, which leave more than 20% of the live answers uncovered, is not applied (`truncated_sample`); a fit on labels that read like the outcome, dated after the documents were changed to show it, is not applied (`labels_read_like_the_outcome`). See [why an epoch has no correction](/calibration/report#why-an-epoch-has-no-correction). Below 100 outcomes each night's fit replaces the one before, so `prior.weight` grows with your outcomes. Each correction's interval is never narrower than your own outcomes support: the engine's numbers make the correction more careful, never more certain. ## Only when it beats the raw numbers Brussle applies a fit only when it does better than the engine's own numbers on held-out outcomes, which the fit did not see. The score is log loss. When the raw numbers score as well or better, Brussle does not apply the fit. Answers then have no `calibrated` object, and the report's `not_fitted` says `raw_better`, with both results in `held_out`. Your answers stay raw and do not get the noise of a fit. The held-out test also gives the Brier score of both (`raw_brier`, `calibrated_brier`), beside log loss and calibration error. The Brier score is the mean squared gap between the number and what happened, from 0 to 1, and lower is better. For a `choice` or `score`, it is the sum of the squared gaps over the options, halved, so a choice of two options reads as a yes-or-no judgment does. Brussle still chooses a fit on log loss alone. ## How sure a corrected number is Each corrected number has `interval`, `{"lower", "upper"}`, its 95% interval. It says how sure the correction is, from the outcomes the fit uses. The recommender's interval and the lift's interval are 95% too. * **Where it is.** `calibrated.interval` is on the number the object shows: `p` for a `bool`, `escape_p` for a `choice`, `score` for a `score`. On a judgment with features, `fit.interval` is on the same number, at the answer's own features. * **It always contains the number.** A probability's interval stays between 0 and 1, and a score's stays between the lowest and highest level. * **More outcomes make it narrower.** Each fit uses all the outcomes so far, so the interval usually narrows from one fit to the next. When a night keeps the current fit, its numbers stay the same and Brussle measures its interval again on all the outcomes so far. The interval is wider for raw values far from most of the fit's outcomes. * **From 1,000 outcomes, it is the interval of the outcomes near this value,** because at `stage: full` the correction follows your outcomes value by value. Where few outcomes are near, the interval is wide (sometimes wider than the `early` fit's interval was at the same value), and the report's [`band`](/calibration/report#the-band) shows where. * **Labelling-queue labels** count as the labels that they are, not as the traffic that they stand for, so a few labels with a large weight give a wide interval. * **Outcomes that run against the engine.** The correction never reverses the engine's order. When higher raw numbers came true less often, the corrected number is the share of events among all the outcomes, and the interval is the interval of that share. * **With `extrapolated: true`** the interval is still there, wider at the edges. But the correction itself is a guess there, and the interval does not know that. * **On a template's tenant,** the interval is never narrower than the tenant's own outcomes support (see [per-customer corrections](/calibration/tenants#the-interval-on-a-tenant)). * **`interval` is `null`** for a fit made before fits recorded their intervals (2026-10-13). The next nightly run adds it, whether it replaces the fit or keeps it. What the interval does not tell you: * How much the engine's answer varies from one call to the next. The engine's raw number has its own variation, which each engine's page under [engines](/engines/index) describes. * That what happens will fall inside it. The interval says how much your outcomes support the correction. It does not show whether your outcomes are a reviewed sample (read the report's `warnings`) or whether the engine changed (see [epochs](/guides/change-a-judgment#epochs)). ## Automate on the bound Automate on the interval, never on the number alone, whatever the stage. Use `interval.lower` to act on high numbers and `interval.upper` to act on low ones. To act when the probability is 0.8 or more, act when `interval.lower` is 0.8 or more, not when `p` is. Queries and subscriptions on a judgment with [features](/calibration/features) filter and rank on `answers..interval.lower` and `answers..interval.upper`, the bounds of `fit.interval`. This query finds the payments whose probability of settling their invoice is 0.8 or more with 95% confidence, the surest first: ```json theme={"theme":{"light":"css-variables","dark":"css-variables"}} { "filters": ["answers.invoice_settlement.interval.lower", "Gte", 0.8], "rank_by": ["answers.invoice_settlement.interval.lower", "desc"], "include": {"answers": ["invoice_settlement"]} } ``` A [subscription](/concepts/subscriptions) with the same filter sends an event when a document starts or stops meeting it. * **A document whose `p` is the engine's has no interval,** so the filter does not select it. `["answers.invoice_settlement.interval.lower", "Exists", false]` selects those documents. * **On a judgment without features,** these fields are `invalid_request`, because readers use the engine's numbers there. Read `calibrated.interval` in your own code instead, or pick a threshold with the [recommender](/calibration/thresholds) and read it with its interval. * **The bounds change when a new fit uses more outcomes.** A query reads the current bounds at once. A subscription checks its filter again when the fit changes (a new `fit.id`), and bounds that narrow under the same fit reach it at the document's next evaluation. ## What the correction can and cannot do * **It is not fine-tuning.** The engine's weights never change. The fit corrects the number the engine gives, from your outcomes and, if the judgment has them, your [features](/calibration/features). * **It makes probabilities agree with your outcomes.** After the correction, a stated 0.9 came true about 90% of the time on your held-out outcomes. The [report](/calibration/report) shows both, before and after. * **It does not change the order of documents, for a judgment without features.** A higher raw probability always maps to an equal or higher corrected one, so the correction cannot make the engine better at telling probable documents from improbable ones. If the right documents are not near the top of the raw ranking, try a narrower question, a different [context recipe](/guides/context-recipes) or counts as features, and measure again. With features the order can change, because your numbers sit beside the engine's answer. * **It cannot separate documents with the same `p`, for a judgment without features.** Jev, for example, gives `p` in steps of 0.01 (each engine's page under [engines](/engines/index) says how its probabilities come). The correction gives each `p` one corrected value, so a threshold selects all of these documents or none. With features, documents with the same `p` usually have different features and get different values. * **It can change a yes or no at 0.5.** Accuracy before and after can differ, and the report shows both. * **It is only as good as the outcomes.** Outcomes from one kind of document, or one part of the range, give a correction that is right there and a guess elsewhere. The report's `not_fitted`, `coverage` and `warnings` say when. * **It is only as current as its last fit.** A fit made before answers changed can be wrong for the new answers (see [when answers are corrected](#when-answers-are-corrected)). ## When answers are corrected Brussle sometimes judges answers again to correct them (for example, after a fault on our side made a judgment read too few related documents). You are not billed for these answers. Each corrected answer is a new evaluation, and the document's history keeps the earlier one. * **Outcomes measure the corrected answer.** An outcome that joined the earlier answer joins the corrected one from then on, also when the document changed after the outcome and, with a horizon, when the correction comes while the earlier answer's window is open. A correction made after that window closed opens the next window, as any new answer does. * **The report changes at once.** `raw` and `calibrated` use the corrected answers from your next request. * **The fit changes at the next run,** which starts within a few minutes after the correction ends. The current fit was made on the replaced answers, so it can trust the corrected ones too much or too little. A judgment with no outcomes has no fit to change. * **The report warns until then.** Each epoch whose fit used answers judged again since with different inputs has `inputs_changed_since_fit`, whose `count` is the number of the fit's outcomes that now join a new answer. The same warning shows when other answers the fit used were judged again with different inputs at the same revision. It clears at the next fit. To fit at once, select **Fit now** or call `POST /namespaces/{ns}/judgments/{name}/calibration/fit` (`fitCalibration`, `fit_calibration`). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.