Skip to main content
The context recipe decides what each answer costs. Once a judgment has enough outcomes, Brussle tests smaller versions of its recipe against them each month at no charge, and suggests the one that costs least per answer with no loss your outcomes can see. Brussle also lists what the recipe can leave out without harm, even when that saves nothing. You decide whether to switch. The smaller recipes are the second group of the alternatives table, and recipe-suggestion shows them too.

​
How tuning works

  • When it runs. A tuning run follows the fit once a version’s correction rests on 100 outcomes, with 20 of each kind. After that it runs at most every 30 days, and again after the engine changes.
  • What saves money. A smaller recipe saves money only where it moves answers into a smaller size class, since fewer tokens in the same size class cost the same.
  • What it tries. Each test recipe makes one change to yours: it leaves out one field or one relation, or halves one last_n, one window, one field’s max_chars or one relation’s last_n. Tuning halves a relation’s last_n only when the relation shows its documents’ fields and has no aggregate and no match on answers or outcomes.
  • How it tests. The test recipes use the contexts already stored with your labelled answers, sent to the same engine on the same terms, and Brussle compares them with the current recipe on the same outcomes. Brussle skips contexts that sent an image you erased.
  • When nothing is cheaper, the run still tests the recipes and lists those that did no worse in no_worse, so even for a recipe already at the smallest size class you see what it can leave out.
  • Cost. Tuning is never billed. It uses only part of your organization’s monthly allowance for free replays, so a refit after the engine changes by itself always has the rest. The alternatives run, which scores engines and named alternatives, is billed as judging, and the allowance does not stop it.

​
Read the suggestion

GET /namespaces/{ns}/judgments/{name}/recipe-suggestion (ns.judgments.recipeSuggestion(name) in TypeScript, recipe_suggestion(name) in Python) returns the last run’s result for the active version.
  • status: ready when there is a suggestion, none when the last run found none, running, or not_run.
  • reason and message say why there is no suggestion.
  • suggestion, when status is ready:
    • recipe: the whole new recipe. removed: what changed, such as “the field state.internal_notes”.
    • definition: the exact body to post as a new version, which is your active version with only context changed. It gives no thresholds, so your current thresholds stay when you activate it.
    • cost_per_answer_before and cost_per_answer_after: the mean number of judgments per labelled answer with each recipe, each counted by its size class times the engine’s weight. unit_reduction is the part saved. On Jev, unit_reduction is 0.75 when answers that were all large (4.0) all become standard (1.0). It is 0.5625 when only three quarters of them do (1.75). On gpt-6-luna the counts are 2.5 times larger, and unit_reduction is the same.
    • held_out: log loss and accuracy with both recipes, each scored on outcomes that the correction did not see. interval is the 95% interval of the change in log loss, where above 0 is worse. A recipe qualifies only when neither log loss nor accuracy is worse beyond noise, and the suggestion is the qualified recipe with the largest unit_reduction. Brussle measures each recipe on the labelled answers it could be tested on, so you can compare two recipes’ reductions, but not always their cost per answer.
    • labels: the labelled answers both recipes were scored on. engine_version: the epoch both were judged in.
  • headline: one sentence, such as “A context recipe 75% cheaper per answer, moving answers to a smaller size class, with accuracy unchanged within noise on 312 labelled answers.”
  • no_worse: once a run has ended, the other test recipes whose held-out log loss was no worse than yours beyond noise, cheaper or not, with the same fields as suggestion but no engine_version or computed_at. Most save nothing on the bill, but each tells you something the recipe can leave out without a loss your outcomes can see. Accuracy is shown, but it is not part of this test.
  • computed_at and next_run_after: when the last run ended, and the earliest time the daily calibration run tries again.

​
Apply a suggestion

The recipe decides what you send to the engine and which changes make Brussle judge again, so Brussle applies nothing for you and the choice stays yours. Create the version and activate it through its shadow report, as for any change to a question.

​
Run tuning now

POST /namespaces/{ns}/judgments/{name}/recipe-tuning (tuneRecipe, tune_recipe) runs this month’s tuning without waiting for the daily calibration run. It returns the result as it is now, which reads running until the run ends. Brussle keeps one run for each version, engine epoch and calendar month, so a second request returns the same run. A version that cannot be tuned yet returns invalid_request with the reason.

​
In the dashboard

The judgment’s Alternatives tab shows the current recipe and the last tuning run, with its status and reason. When a suggestion is ready, the tab shows what the suggestion changes, the cost per answer before and after, the held-out log loss and accuracy with the interval, and the labels they rest on. After a run it also lists the recipes that were no worse, with what each leaves out and its scores. Create version posts the suggestion’s definition as a new, inactive version and starts its shadow report on the Overview, where you confirm or cancel it. Run now starts this month’s run.

​
Plans

Recipe tuning is on Team and Scale. On Developer, recipe-suggestion shows the headline and the numbers but withholds recipe, removed and definition (in suggestion and in no_worse), with plan_required beside them, and recipe-tuning returns plan_required.