Skip to main content
The context recipe decides what each answer costs. Once a judgment has enough outcomes, Brussle tests smaller versions of its recipe against them each month, at no charge, and suggests the one that costs least per answer with no loss your outcomes can see. It also lists what the recipe can leave out without harm, even when that saves nothing. You decide whether to switch. The smaller recipes are the second group of the alternatives table, and recipe-suggestion shows them too.

​
How tuning works

  • When it runs. Once a version’s correction rests on 100 outcomes, with 20 of each kind, a tuning run follows the fit, then runs at most every 30 days, and again after the engine changes.
  • What saves money. A smaller recipe saves money only where it moves answers into a smaller size class. Fewer tokens in the same size class cost the same.
  • What it tries. Each test recipe makes one change to yours: it leaves out one field or one relation, or halves one last_n, one window, one field’s max_chars or one relation’s last_n. A relation’s last_n is halved only when the relation shows its documents’ fields and has no aggregate and no match on answers or outcomes.
  • How it tests. The test recipes use the contexts already stored with your labelled answers, sent to the same engine on the same terms, and are compared with the current recipe on the same outcomes. Contexts that sent an image you erased are skipped.
  • When nothing is cheaper, the run still tests the recipes and lists those that did no worse in no_worse, so even for a recipe already at the smallest size class you see what it can leave out.
  • Cost. Tuning is never billed. It uses only part of your organization’s monthly allowance for free replays, so a refit after the engine changes by itself always has the rest. The alternatives run, which scores engines and named alternatives, is billed as judging, and the allowance does not stop it.

​
Read the suggestion

GET /namespaces/{ns}/judgments/{name}/recipe-suggestion returns the last run’s result for the active version. It is ns.judgments.recipeSuggestion(name) in TypeScript and recipe_suggestion(name) in Python.
  • status: ready when there is a suggestion, none when the last run found none, running, or not_run.
  • reason and message: why there is no suggestion:
  • suggestion, when status is ready:
    • recipe: the whole new recipe. removed: what changed, such as “the field state.internal_notes”.
    • definition: the exact body to post as a new version: your active version with only context changed. It gives no thresholds, so your current thresholds stay when you activate it.
    • cost_per_answer_before and cost_per_answer_after: the mean number of judgments per labelled answer, each counted by its size class times the engine’s weight, with each recipe. unit_reduction is the part saved. On Jev, it is 0.75 when answers that were all large (4.0) all become standard (1.0), and 0.5625 when only three quarters do (1.75). On gpt-6-luna the counts are 2.5 times larger and unit_reduction is the same.
    • held_out: log loss and accuracy with both recipes, each outcome scored with a correction made without it. interval is the 95% interval of the change in log loss; above 0 is worse. A recipe qualifies only when neither log loss nor accuracy is worse beyond noise. The suggestion is the qualified recipe with the largest unit_reduction. Each recipe is measured on the labelled answers it could be tested on, so you can compare two recipes’ reductions, but not always their cost per answer.
    • labels: the labelled answers both recipes were scored on. engine_version: the epoch both were judged in.
  • headline: one sentence, such as “A context recipe 75% cheaper per answer, moving answers to a smaller size class, with accuracy unchanged within noise on 312 labelled answers.”
  • no_worse: once a run has ended, the other test recipes whose held-out log loss was no worse than yours beyond noise, cheaper or not, with the same fields as suggestion but no engine_version or computed_at. Most save nothing on the bill; each tells you something the recipe can leave out without a loss your outcomes can see. Accuracy is shown, but is not part of this test.
  • computed_at and next_run_after: when the last run ended, and the earliest time the daily calibration run tries again.

​
Apply a suggestion

Brussle applies nothing for you: the recipe decides what you send to the engine and which changes make Brussle judge again, so it stays your choice. Create the version and activate it through its shadow report, as for any change to a question:

​
Run tuning now

POST /namespaces/{ns}/judgments/{name}/recipe-tuning (tuneRecipe, tune_recipe) runs this month’s tuning without waiting for the daily calibration run. It returns the result as it is now, running until the run ends. There is one run for each version, engine epoch and calendar month, so a second request returns the same run. A version that cannot be tuned yet returns invalid_request with the reason.

​
In the dashboard

The judgment’s Alternatives tab shows the current recipe and the last tuning run, with its status and reason. When a suggestion is ready, it shows what the suggestion changes, the cost per answer before and after, the held-out log loss and accuracy with the interval, and the labels they rest on. After a run it also lists the recipes that were no worse, with what each leaves out and its scores. Create version posts the suggestion’s definition as a new, inactive version, and starts its shadow report on the Overview, where you confirm or cancel it. Run now starts this month’s run.

​
Plans

Recipe tuning is on Team and Scale. On Developer, recipe-suggestion shows the headline and the numbers, and withholds recipe, removed and definition (in suggestion and in no_worse), with plan_required beside them. recipe-tuning returns plan_required.