> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brussle.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Change a question or engine safely

> Every answer records the question version and engine version behind it. See the cost before a rerun, compare versions side by side, and know when the engine changed by itself.

A judgment's definition has versions, as a database schema does. This page serves one problem: a change to the question or to the engine must not mix old answers with new answers without notice.

The engine is the AI model that makes the answers. Each answer records the version of the judgment that made it, in `judgment_version`. It also records the engine version, in `engine_version`. For example, a payment's answer for [`settles`](/guides/entity-matching) can read:

```json theme={"theme":{"light":"css-variables","dark":"css-variables"}}
{"settles": {"type": "choice", "value": "inv_2043", "freshness": "fresh", "judgment_version": 3, "engine": "jev", "engine_version": "current+2026-09-24.1"}}
```

The example shows only some fields. See [answers](/concepts/answers).

What you can rely on:

* An answer from a version that asks differently reads `stale` after you activate a new version. A query for current answers leaves it out.
* Activation judges nothing again. A [backfill](/concepts/judgments#backfill) judges the old answers again, and it shows its cost before it starts.
* A shadow report compares the new version with the active version on the same documents, before you switch.
* When the engine changes by itself, each new answer records the new engine version. The engine change never makes an answer stale.

## What a new version does

To change a judgment's question, criteria, context recipe, engine or `horizon`, post the definition again under the same name. Brussle creates version n+1. The new version stays inactive until you activate it. See [versions](/concepts/judgments#versions).

If the create's cost estimate takes more than 20 seconds, the create answers `202` with a `judgment_create` job. When the job is `done`, its `result` holds the response of the create, for example the new version. The SDKs wait for the job for you. See [when a change takes long](/concepts/judgments#when-a-change-takes-long).

Activate it with `POST /namespaces/{ns}/judgments/settles/activate`:

```json theme={"theme":{"light":"css-variables","dark":"css-variables"}}
{"version": 4}
```

* **A version that changes only thresholds** becomes active at once. It judges as the active version does. Thus, the answers of the earlier version stay `fresh`.
* **A version that asks differently** starts a [shadow report](#change-a-question-safely-with-a-shadow-report). The switch happens when you confirm it.

After the switch, Brussle uses the new version for each new evaluation. Activation itself judges nothing again. Each answer from an earlier version that asks differently reads `stale`, with `stale_reason` `older_version`:

```json theme={"theme":{"light":"css-variables","dark":"css-variables"}}
{"settles": {"type": "choice", "value": "inv_2043", "freshness": "stale", "stale_reason": "older_version", "judgment_version": 3}}
```

* **Its numbers stay.** The answer keeps the numbers that version 3 made, until Brussle judges the document again.
* **A query for current answers leaves it out.** `answers: "fresh_only"` removes the row. A filter on `freshness` sees `stale`.
* **A subscription still matches it on its numbers.** A subscription does not filter on freshness. Activation sends no subscription event for the version alone. Thresholds that come with the version act as any change to thresholds does.

What judges the answer again depends on the judgment's [freshness policy](/guides/freshness-policies):

| Policy | The answer is judged again |
| - | - |
| `on_change` or `manual` | When the document changes, or when a backfill runs |
| `periodic` | At the judgment's next run, or when a backfill runs |
| `on_read` | When something reads it. The read shows `pending`, and Brussle judges the document. |

### Judge the old answers again with a backfill

A backfill judges each document in scope that has no successful answer from the active version for its current revision. Send `POST /namespaces/{ns}/judgments/settles/backfill` without `confirm` first. Brussle returns the estimate and does nothing else:

```json theme={"theme":{"light":"css-variables","dark":"css-variables"}}
{"estimate": {"documents": 1204332, "tokens": 2408664000, "judgment_units": 1204332, "cost_usd": 301.08, "duration_s": 116940}}
```

The numbers are illustrative, for a judgment on jev. On gpt-6-luna, the same backfill counts 2.5 times as many judgments. Send the same request with `"confirm": true` to start the backfill as a job. See [backfill](/concepts/judgments#backfill) for the fields and the job.

On a large namespace, Brussle can take more than 20 seconds to count the documents. Then the backfill request, with or without `confirm`, answers `202` with a `backfill_start` job. When that job is `done`, its `result` holds the estimate, or the `job_id` of the backfill that started. The SDKs wait for the job for you. See [when a change takes long](/concepts/judgments#when-a-change-takes-long).

A backfill skips the answers from a version that changed only thresholds. These answers read `fresh`. Thus, after such a version, the estimate is zero.

## Change a question safely with a shadow report

A shadow report shows what a new version of a judgment would change, before you switch to it.

When version 4's engine or definition is different from the active version, the activate call returns `202` with a `shadow` job in `awaiting_confirm`. The job judges a random sample of 1,000 documents under version 4. If there are fewer documents, it judges all of them. While the job samples, `report` is `null`, and `progress.documents_done` counts the sampled documents. When the job is done, the report compares the two versions on the same documents:

```json theme={"theme":{"light":"css-variables","dark":"css-variables"}}
{
  "id": "job_01j8zq3k5v9w2x4y6z8a0b1c2d",
  "type": "shadow",
  "status": "awaiting_confirm",
  "namespace": "acme/prod/tenant_123",
  "judgment": "settles",
  "version": 4,
  "progress": {"documents_done": 1000},
  "spend_usd": 0,
  "created_at": "2026-10-02T09:14:03Z",
  "updated_at": "2026-10-02T09:31:47Z",
  "report": {
    "documents": 1000,
    "threshold_flips": {"unmatched": {"to_true": 31, "to_false": 6}},
    "recompute": {"documents": 1204332, "tokens": 2408664000, "judgment_units": 1204332, "cost_usd": 301.08, "duration_s": 116940}
  }
}
```

The values are illustrative. The example leaves out `current` and `candidate`.

* **`current`** is the active version's answers. **`candidate`** is the new version's answers. For `bool` and `score`, each side has a `mean` and a 10-bin `histogram` of `p` or `score`. For `choice`, each side has `dist`, the mean probability of each option. Thus, you can read the change option by option.
* **`threshold_flips`** counts, for each named threshold, the documents that would go from false to true and from true to false. The new side uses the thresholds that will apply when the version is active. In the example, 31 more payments would wait for a person in the unmatched queue.
* **`recompute`** estimates a backfill of each document under the new version: documents, tokens, judgments counted by size class times the weight of the new version's engine, cost and duration.

Then decide:

* **`POST /jobs/{id}/confirm`** (`db.jobs.confirm(id)`) switches to the new version. Confirm once the report is ready. A confirm before the report exists returns `conflict` and changes nothing. The job shows `running`, then `done` about a second later, when Brussle has committed the switch.
* **`POST /jobs/{id}/cancel`** keeps the active version as it is.
* **`{"version": 4, "force": true}`** on activate skips the report and switches immediately. `activate: true` on a create of a version does the same. A [composite judgment](/guides/composite-judgments#activate-it-measured-on-your-labels) is the exception. Its shadow job fits it on your labels, so the shadow job always runs.

Brussle writes shadow evaluations to the evaluation log with `shadow: true`. They never make answers. They never count toward calibration. Brussle does not bill them.

Thresholds that you gave with the new version replace the current thresholds when the version becomes active.

In the dashboard, the **Overview** of the judgment's page shows the report, with **Confirm**, **Cancel** and **Force**.

For a judgment on a [template](/guides/templates#change-the-question-for-every-tenant), activate the new version on the prefix path. You get one shadow report, with a sample from all the tenants.

## Find answers by version

A query can filter on the version that made each answer:

* **`answers.<judgment>.judgment_version`** is an integer. Compare it with `Eq`, `NotEq`, `Lt`, `Lte`, `Gt`, `Gte`, `In` or `NotIn`, or test it with `Exists`.
* **`answers.<judgment>.engine_version`** is a string. Compare it with `Eq`, `NotEq`, `In` or `NotIn`, or test it with `Exists`. For an engine version that is not pinned, such as jev `current` or gpt-6-luna `current`, it holds the label of the [epoch](#epochs). The label does not name the engine, and two engines can have the same label. A judgment changes its engine only in a new version, so filter on `judgment_version` too when a judgment has used more than one engine.

For example, this query finds the payments whose `settles` answer is from a version before version 4:

```json theme={"theme":{"light":"css-variables","dark":"css-variables"}}
{
  "filters": ["answers.settles.judgment_version", "Lt", 4],
  "top_k": 100,
  "include": {"answers": ["settles"], "state": false}
}
```

Send it to `POST /namespaces/{ns}/query`. Use the same filter on `engine_version` to compare the answers of two epochs side by side. See [find the answers of one version](/concepts/queries#find-the-answers-of-one-version).

## When the engine changes

### Epochs

An epoch is a period in which the engine's behaviour stays the same. The `engine_version` of an evaluation records its epoch.

* **An exact engine version** is one epoch for its full life.
* **An engine version that is not pinned**, such as Jev `current`, starts a new epoch each time its behaviour changes. The label is like `current+2026-09-24.1`. See the [engines](/engines/index).

A new epoch never makes an answer stale, and it judges nothing again. Answers from before keep their epoch in `engine_version`. Each new answer records the new epoch.

You can see that the engine changed in these places:

* **An event.** Each judgment whose active version uses the changed engine version gets one `judgment.engine_changed` event for each new epoch. If the conformance run on the new epoch does worse than the last run that passed, each of these judgments also gets one `judgment.engine_regressed` event. A judgment that a template defines gets its events on the template. See the [event types](/guides/events-feed#event-types).
* **The answers.** New answers have a new `engine_version`. Filter on it to find them.
* **The calibration report.** `lift.model_changed` gives the earlier epoch, the current epoch and the day that the current epoch started. See [read the lift](/guides/measure-improve-tune#read-the-lift).
* **The Calibration tab.** It says that the engine changed while Brussle fits the new epoch.

Brussle fits calibration for each judgment version and each epoch, because a different engine version needs a different fit. After the engine changes by itself, the new epoch starts with no outcomes of its own. Brussle fits it again from the outcomes that you already posted. This is for a change in the behaviour of `current`. When you move a judgment to another engine yourself, see [change the engine yourself](#change-the-engine-yourself).

### After the engine changes

Some minutes after an engine version that is not pinned, such as Jev `current`, changes, Brussle fits each judgment with outcomes again for the new engine version. It uses a **replay**:

1. Brussle takes the stored contexts of the answers that your most recent outcomes labelled, up to a fixed number.
2. The same engine judges these contexts again.
3. Each new answer, with its old outcome, is a sample of the new epoch.

You do not have to do anything. The replay runs on each plan. [Calibration](/concepts/calibration#when-the-engine-changes) explains what Brussle sends to the engine.

What you see:

* **The Calibration tab.** For the new epoch, while the replay runs, it says "Re-fitting after the engine changed on 2026-10-02: 400 of 812 outcomes replayed so far." Then it says "Re-fitted on 812 replayed outcomes after the engine changed on 2026-10-02."
* **The report.** The new epoch has `fit_source: "replay"` and its `replay` progress. Its own `outcomes` stay 0 until you post new outcomes.
* **Answers.** Until Brussle makes the replay's fit, usually in less than an hour, answers under the new epoch use the previous epoch's calibration with `"from_previous_epoch": true`. For a judgment without features, the previous epoch's calibration lasts for 30 days at most. A judgment with features keeps the previous epoch's fit until the new epoch has its own.
* **Evaluation history.** Replays show in a document's evaluation history with `replay_of`, which names the evaluation that they replayed. Replays never change an answer. Brussle never bills them.
* **Outcomes.** Continue to post outcomes as usual. When the new epoch has 100 of its own, with 20 of each kind, its fit uses only them, and `fit_source` becomes `outcomes`.

A replay can stop early (`status: "stopped"`). Then Brussle fits the new epoch on what it replayed, if that meets the minimums. If not, Brussle fits the new epoch from its own outcomes as they arrive. The `reason` tells why the replay stopped:

* `cost_cap`: your organization reached this month's replay limit. If the epoch still has no fit next month, the replay runs again on the outcomes that it did not replay.
* `superseded`: the engine changed again. The newer epoch has its own replay.

Fits after an engine change come first. [Recipe tuning](/guides/measure-improve-tune#let-your-outcomes-tune-the-recipe) uses only part of the monthly limit. Thus, these fits always have the rest.

### Change the engine yourself

The engine is part of the definition. To pin an exact engine version, or to move to a different engine, post a new version with the new `engine`. Then activate it through its [shadow report](#change-a-question-safely-with-a-shadow-report), as for any change to the question. Answers from the earlier version read `stale` with `older_version` until Brussle judges them again.

For example, to move `settles` from jev `current` to gpt-6-luna `current`, post version 4 with `"engine": {"name": "gpt-6-luna", "version": "current"}` and the same question and recipe. These things change with the engine:

* **Cost.** Each judgment counts its size class times the [weight](/pricing#judgments) of the new engine: 1 on jev, 2.5 on gpt-6-luna. Each engine also counts tokens in its own way, so some documents can change size class. The shadow report's `recompute` prices a backfill on the new engine.
* **What the engine reads.** gpt-6-luna reads the [images](/guides/images) that the recipe selects. jev gets only their references. Thus, the context, its size class and the answers can all change.
* **The answers.** The shadow report compares the raw answers of the two engines on the same documents. It does not compare calibrated values.
* **Calibration starts again.** The new version has no fit. Its answers have no `calibrated` until it has 100 outcomes of its own, with 20 of each kind. Brussle does not replay your earlier outcomes through the new engine.
* **Epochs.** Each engine numbers its own epochs, so jev and gpt-6-luna can have the same label, such as `current+2026-10-07.1`. To find the answers of the new engine, filter on `judgment_version`, not on `engine_version` alone.
* **Events.** The judgment gets `judgment.engine_changed` events for the new engine's epochs, and none for the old engine's.


## Related topics

- [Measure against outcomes](/guides/measure-improve-tune.md)
- [What Brussle is](/index.md)
- [Judgments](/concepts/judgments.md)
- [Import existing data](/guides/import-existing-data.md)
- [Engines](/engines/index.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.