Skip to main content
If you run the same judgments for many customers, each in its own namespace under a template, every customer is corrected from its first answer by the correction pooled across all of them. On Team and Scale, a customer with enough outcomes of its own also gets its own correction, held toward the shared one until its outcomes clearly say otherwise, and kept or replaced by the same rule as any fit. The report says, for each customer, which correction it reads and why. Here a customer of yours is a tenant: a namespace that inherits the judgment from a template’s prefix.

​
The pooled correction

Tenants post outcomes on their own namespace paths, never on the prefix. The correction on the prefix pools them: one fit for each template version and engine epoch, fitted each night across every tenant’s outcomes. The template’s run makes every tenant’s fit at the same time.
  • A tenant without its own fit reads the pooled fit, so a tenant you create today is corrected from its first answer. The pooled fit was made on other tenants’ outcomes, so it is not yet measured on this one. Once the tenant posts outcomes, the report on its own path measures what it reads against them.
  • Tenants under a non-production prefix, such as acme/staging/, read the pool but add nothing to it and get no fit of their own, unless the template itself is under that prefix.
  • Settings live on the template. Set implicit negatives, pins, promotions and named runs on the prefix path. A tenant cannot set calibration, promote or make a named run (invalid_request). Neither the prefix nor a tenant can set outcome rules.
  • The list of fits on a tenant’s path shows the template’s fits, which are what its answers read.
  • A new version starts each tenant on that version’s pool again. Tenants keep following the template, and you do not detach anything.
  • A composite keeps one pooled combiner for all tenants.

​
Each tenant grows its own correction (Team and Scale)

A tenant’s data can differ from the pool’s: another language, a stricter review team, another base rate. On Team and Scale, a tenant with 50 outcomes of its own in an epoch, 15 of each kind, also gets its own fit. Below that, it reads the pool.
  • It starts at the pool’s fit and moves toward the tenant’s own outcomes as they grow: n / (n + 100) of the way on n outcomes, one third at 50 and a half at 100. The report shows this as weight, and each corrected answer as prior with kind: pool.
  • It is used only when it is clearly better than what the tenant would read otherwise, the pool or raw, on the tenant’s held-out outcomes: by more than two standard errors. Until then the tenant reads the pool, or raw if the pool does not beat raw.
  • It is kept by the same rule as the pool’s. Each night the tenant is fitted again, and the new fit replaces the one it reads only when it is clearly better on the tenant’s own outcomes since, and moves its numbers by at least 0.01. When the pool beats the tenant’s current fit by the same margin on those outcomes, the tenant goes back to the pool. See the rule.
  • When the pool’s fit changes, the pool part of each tenant’s blend changes with it. The pool is tested on every tenant’s outcomes, where one tenant is a small part. The next night’s report of each tenant shows the new blend against the pool, in tenant.current.since.
  • A promote on the template (POST .../calibration/fit with {"promote": true} on the prefix path) makes the newest fit current for the pool and every tenant.
  • Plans. Below Team, each tenant reads the pool. After a downgrade takes effect, the next fit puts each tenant back on the pool. After an upgrade, the next fit gives tenants their own fits.

​
Its own weights over features

On a bool with features, a tenant’s own fit has its own weights over the engine’s p and the features, from 100 of its own outcomes with 20 of each kind. A count of open tickets can mean less in one tenant than across the pool. Each weight starts at the pool’s and moves toward what the tenant’s outcomes show, by the same n / (n + 100), and is kept by the same rule. A feature that none of the tenant’s documents has keeps the pool’s weight. The tenant’s p is then its own fit over the answer’s raw.p and features, and the answer’s fit has "tenant": true and the tenant’s own id. A choice or score with features reads the pooled fit in every tenant.

​
Which readers are judged again

On a judgment with features, readers act on p. When one tenant’s own fit changes, Brussle judges again the readers in that tenant only. When the pooled fit changes, it judges again the readers in the tenants that read the pool. A tenant that reads its own fit is not changed by the pool.

​
The interval on a tenant

When a tenant reads its own fit blended with the pool, the interval of each corrected number is the same blend of its own fit’s interval and the pool’s, widened to hold its own fit’s interval. The pool pulls the number toward the other tenants, and the interval does not hide that: it is never narrower than the tenant’s own outcomes support. A tenant that reads the pool reads the pool’s interval. On a bool with features, a tenant that reads its own fit reads its own fit’s interval. On a tenant that reads a fit of its own, the corrected number blends the tenant’s fit with the template’s fit that fit names, and a fit’s formula is the template’s, not the tenant’s own.

​
Pin a fit for every tenant

A pin holds one pooled fit for one version and epoch. Set it on the prefix path, with PATCH and calibration, as on a namespace.
  • Each tenant reads the pinned fit for that version and epoch, also a tenant that had its own. No tenant gets its own fit for it while the pin holds, and each tenant’s report shows tenant.reason pinned.
  • Other versions and epochs are not changed. Tenants keep their own fits there.
  • On a bool with features, the readers are those of every tenant that inherits the judgment. The PATCH returns their cost in downstream, from up to 100 tenants and scaled to all of them, as for a threshold change. With confirm, one resync job judges again the readers whose view of the judgment changes.
  • When you lift the pin, Brussle brings the next run forward. On Team and Scale, that run can give tenants their own fits again.

​
Read the report on a tenant

On the prefix path, the report and threshold recommendations use the pooled outcomes. On a tenant’s path, they use that tenant’s outcomes, and each epoch’s tenant says what the tenant reads:
On a bool with features, tenant also has method and weights:
  • applied is tenant_shrunk (the tenant’s own fit, held toward the pool’s), pooled or raw.
  • weight is how far the tenant’s own fit moves from the pool’s toward its own outcomes, n / (n + 100).
  • method, on a bool with features only, is features: the tenant has its own weights over the features.
  • weights, on a bool with features only, gives each input’s weight under the pool’s fit and the tenant’s own, side by side: first the engine’s p (engine), then each feature. A weight is per standard deviation of the input across the template, so you can compare inputs. The difference between the columns is what the tenant’s own outcomes show: in the example, open tickets count about twice as much for this tenant as for the pool. null while the tenant has no fit of its own.
  • reason says why:
    • With its own fit: tenant_better, pooled_better or raw_better.
    • Before that: too_few_outcomes, awaiting_fit (the template’s next fit makes one, within a day, or sooner when you ask for a fit) or plan_required.
    • non_production: the tenant is under a non-production prefix. It reads the pool and adds nothing to it.
    • pooled_only: a choice or score with features. Each tenant reads the pool.
    • pinned: the template pins a fit for the version and epoch.
  • tenant.current and tenant.challenger: the tenant’s own fit and the night’s, on the tenant’s own outcomes since, against the pool (fallback: pooled). See read the current fit and the challenger.
The epoch’s stage, held_out, not_fitted and calibrated then describe the fit the tenant reads, measured on its own outcomes. On a tenant with its own fit, held_out_metrics is null and the tenant’s own test gives no holdout or standard errors. lift is null on every tenant: the template’s report has the pool’s lift. On the prefix path, each epoch’s tenants counts the tenants with outcomes in the epoch, and how many of them read their own fit, the pool or raw.