> ## Documentation Index > Fetch the complete documentation index at: https://docs.brussle.com/llms.txt > Use this file to discover all available pages before exploring further. # Append observed outcomes or labelled examples > Outcomes are append-only. They are joined when calibration, the report or a recommendation reads them, never when posted, so an evaluation or an outcome that arrives late still counts. An outcome that joins nothing is kept, and the calibration report counts it in `unmatched_outcomes`. Outcomes feed the calibration report, the `calibrated` answer fields and threshold recommendations; see [calibration](/concepts/calibration). An outcome whose value does not match the judgment's type is `invalid_request`; one for a judgment the namespace does not have is `not_found`. - **Labelled examples** are outcomes with the default horizon of `0s`: write the documents, post their labels, and each label is joined to the document's evaluation of the revision current at `observed_at`. This measures accuracy before going live. - **Real-world results** use a horizon, such as "churned within 30 days" with `horizon: "30d"`. An answer predicts the event within the horizon after it was made. A document's first answer opens the window `(t, t + horizon]`, answers made inside an open window open none, and the first answer after it closes opens the next. An outcome labels the window its `observed_at` falls in; several in one window count once, a `true` winning. A `false` that falls in no window, posted once the period is over, labels the latest window that closed by its `observed_at`. Once a `true` is observed, later windows of the document count for nothing until it is deleted and created again: the event already happened. An outcome observed after the new life's first answer never labels a window from before the delete. An outcome observed while its document is deleted joins nothing, whatever the horizon. A document written again remembers its id's last 8 deletions, so this holds however late the outcome is posted. It needs the deletion to still be recorded when the outcome is posted or read, or when the id is written again. Post what did not happen as well as what did: calibration needs at least 20 outcomes of each class. For a `bool` judgment with a horizon, the `outcomes.implicit_negatives` setting counts each closed window with no outcome as `false`. When the truth already arrives in your writes, the `outcomes.rules` setting derives outcomes from them instead; a posted outcome beats a derived one in the same window. A label for a document the labelling queue drew carries its `queue_item_id`; it counts as `source: queue`. An id whose draw is gone, or that names another document, is `invalid_request`; one whose 7-day lease has run out is `conflict`. An answer keeps one queue label, the latest recorded, so a retry or a correction counts once. Outcomes belong to a namespace, not a template prefix: post them on each namespace's own path. ## OpenAPI ````yaml /api-reference/openapi.yaml post /namespaces/{ns}/outcomes openapi: 3.1.0 info: title: Brussle API version: '1' license: name: Proprietary identifier: LicenseRef-Proprietary description: | The `/v1` contract. It is locked: changes within `/v1` are additive only, and anything breaking is `/v2`. - **Base URL.** `https://api.brussle.com/v1`. - **Evolution.** Response objects may gain fields and enums may gain values. Clients must ignore what they do not know. Request objects reject unknown fields. - **Path segments.** Namespace names (`{ns}`) and document ids (`{id}`) may contain `/`. Send it percent-encoded as `%2F`, so that `acme/prod/tenant_123` is `acme%2Fprod%2Ftenant_123`. - **Templates.** The judgment routes also take a namespace prefix ending in `/*`, such as `acme%2Fprod%2F*`. A judgment created there is a template: every namespace under the prefix inherits it, including ones created later. The most specific matching prefix wins. - **Thresholds are settings.** Named thresholds belong to the judgment, not to a version. They are evaluated at read time against the raw fields, so a change applies at once to every answer, with no recompute and no new version. - **Entity judgments.** A judgment can read the documents that point at the judged one, through `context.related`, and apply only to documents matching `applies_to`. Its answers carry a `watermark`, and creating one that runs `on_change` returns a replay estimate of its monthly cost until you confirm. - **Referenced documents.** A relation can also read the one document the judged document points at, through `join: {theirs: "id", mine: "attributes."}`. A change to what the relation renders of that document re-judges the judged documents that point at it and are inside `freshness.fanout.scope`: a fan-out. - **Relations.** A relation can read plain judgments' answers (banded or cut, never summed), or be a blocking relation over the documents that share a key, which a `choice` can choose among with `options.from`. A recipe can read the document as the judgment last saw it (`context.previous`). A judgment that can fan out confirms its rolling limits once, at create; past them, work is deferred, reads `stale` with an event, and catches up on its own. Nothing waits for a confirm at runtime. Groups keep aggregates per key value for reporting. - **Idempotency.** Every write operation is idempotent by construction: a retried append adds nothing while its values are still in the array, and a retried upsert or patch sets the same content again. Every request that changes something also takes an `Idempotency-Key`, so that creates, activations, backfills, draws and the rest can be retried safely too: a retry with the key gets the first response back, verbatim and marked `Idempotent-Replayed: true`, instead of running again (see the parameter). - **Errors.** Every error is the `Error` envelope; the HTTP status follows the code (see each response). - **Rate limits.** Every response to a request whose key was accepted carries `X-RateLimit-Limit` and `X-RateLimit-Remaining`, which count the key's requests a second; a request refused before its key is checked (`unauthorized`, or a route that does not exist) carries neither. A `rate_limited` response adds `Retry-After`, except a daily allowance that is used up, which says when it renews in `details.resets_at` instead. Every `503` carries `Retry-After` too. Reads of one namespace also have a concurrency limit: past 32 gets and queries in flight at once, one more is `rate_limited` with `Retry-After` and `details.concurrent_reads`, whatever the rate-limit headers show. - **Usage.** Every response that bills carries `usage`. Judging is billed per judgment, engine-neutral, by size class: one judgment answered counts by the tokens of its compiled context and its question together, 1 if standard (up to 2,000 tokens), 4 if large (up to 8,000) and 16 if extra-large (up to 32,000, the engine maximum). No response ever carries an engine's price. servers: - url: https://api.brussle.com/v1 security: - apiKey: [] tags: - name: Namespaces description: Namespaces, their settings and cache warming. - name: Documents description: Writes, point reads and queries. - name: Judgments description: >- Judgment definitions, versions, settings, activation, backfill and template detach. - name: Outcomes and calibration description: | Post what actually happened to judged documents, and read how well answers match it: the calibration report, calibrated answers and threshold recommendations. See [calibration](/concepts/calibration). - name: Groups description: > Groups keep aggregates per value of a key attribute, a group-by you declare for reporting; a template's tenant summary publishes each tenant's groups, banded, as a document the platform can judge. - name: Jobs description: >- Backfill, shadow, periodic, deletion, reference index, evaluation export, resync, group build, discovery and simulation jobs. - name: Engines description: The engine registry. - name: Organization description: | Settings of the whole organization: prefixes marked non-production, which the tenant fee and template calibration pools leave out. - name: Subscriptions description: | Saved queries that send an event when a document starts or stops matching: the query's filter grammar, on a namespace or a template prefix, with the fields each event carries. - name: Webhooks description: | Webhook endpoints, the events feed, the delivery log, redelivery and recovery. Every event type and its body is under `webhooks`. Events are delivered at least once, signed as Standard Webhooks, and kept for 30 days. Webhooks are on every plan, and deliveries are not billed; plans set how many endpoints an organization has. paths: /namespaces/{ns}/outcomes: parameters: - $ref: '#/components/parameters/Namespace' post: tags: - Outcomes and calibration summary: Append observed outcomes or labelled examples description: | Outcomes are append-only. They are joined when calibration, the report or a recommendation reads them, never when posted, so an evaluation or an outcome that arrives late still counts. An outcome that joins nothing is kept, and the calibration report counts it in `unmatched_outcomes`. Outcomes feed the calibration report, the `calibrated` answer fields and threshold recommendations; see [calibration](/concepts/calibration). An outcome whose value does not match the judgment's type is `invalid_request`; one for a judgment the namespace does not have is `not_found`. - **Labelled examples** are outcomes with the default horizon of `0s`: write the documents, post their labels, and each label is joined to the document's evaluation of the revision current at `observed_at`. This measures accuracy before going live. - **Real-world results** use a horizon, such as "churned within 30 days" with `horizon: "30d"`. An answer predicts the event within the horizon after it was made. A document's first answer opens the window `(t, t + horizon]`, answers made inside an open window open none, and the first answer after it closes opens the next. An outcome labels the window its `observed_at` falls in; several in one window count once, a `true` winning. A `false` that falls in no window, posted once the period is over, labels the latest window that closed by its `observed_at`. Once a `true` is observed, later windows of the document count for nothing until it is deleted and created again: the event already happened. An outcome observed after the new life's first answer never labels a window from before the delete. An outcome observed while its document is deleted joins nothing, whatever the horizon. A document written again remembers its id's last 8 deletions, so this holds however late the outcome is posted. It needs the deletion to still be recorded when the outcome is posted or read, or when the id is written again. Post what did not happen as well as what did: calibration needs at least 20 outcomes of each class. For a `bool` judgment with a horizon, the `outcomes.implicit_negatives` setting counts each closed window with no outcome as `false`. When the truth already arrives in your writes, the `outcomes.rules` setting derives outcomes from them instead; a posted outcome beats a derived one in the same window. A label for a document the labelling queue drew carries its `queue_item_id`; it counts as `source: queue`. An id whose draw is gone, or that names another document, is `invalid_request`; one whose 7-day lease has run out is `conflict`. An answer keeps one queue label, the latest recorded, so a retry or a correction counts once. Outcomes belong to a namespace, not a template prefix: post them on each namespace's own path. operationId: appendOutcomes parameters: - $ref: '#/components/parameters/IdempotencyKey' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/OutcomesRequest' example: outcomes: - document_id: t_123 judgment: needs_escalation value: true observed_at: '2026-10-15T00:00:00.000Z' responses: '202': description: The outcomes were appended. headers: X-RateLimit-Limit: $ref: '#/components/headers/RateLimitLimit' X-RateLimit-Remaining: $ref: '#/components/headers/RateLimitRemaining' '400': $ref: '#/components/responses/InvalidRequest' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/OutcomeConflict' '413': $ref: '#/components/responses/TooLarge' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/Internal' components: parameters: Namespace: name: ns in: path required: true description: The namespace name, with any `/` sent as `%2F`. schema: $ref: '#/components/schemas/NamespaceName' IdempotencyKey: name: Idempotency-Key in: header description: | One key per logical request, reused only on its retries. A key belongs to one request: within your organization, the same method, path, query and body. For 24 hours after a successful response, a request with the key and the same body gets that response back verbatim, with `Idempotent-Replayed: true`, and runs nothing. Only a successful response is kept, so the retry of a request that failed runs again. While the first request runs or its response is kept, the key with a different request is `idempotency_key_reused` (422). A request sent while one with its key is still running is `rate_limited` with `Retry-After: 1`, without running: retry it to get the first one's response. In the rare case the key can't be checked, the request runs as if it had none. schema: type: string minLength: 1 maxLength: 255 schemas: OutcomesRequest: type: object additionalProperties: false required: - outcomes properties: outcomes: type: array minItems: 1 items: $ref: '#/components/schemas/Outcome' NamespaceName: type: string description: >- Up to 256 bytes. `/` separates levels of the hierarchy, as in `acme/prod/tenant_123`. Never exactly `.` or `..`, which a URL path can't carry. pattern: ^(?!\.\.?$)[A-Za-z0-9._:/-]+$ minLength: 1 maxLength: 256 Outcome: type: object additionalProperties: false required: - document_id - judgment - value - observed_at properties: document_id: $ref: '#/components/schemas/DocumentId' judgment: $ref: '#/components/schemas/Name' value: description: >- A boolean, an option value or a level value, matching the judgment type. type: - boolean - string - integer observed_at: $ref: '#/components/schemas/Timestamp' description: >- When the outcome happened. For a labelled example, when the label applies, usually now. queue_item_id: type: string description: >- The labelling-queue item this labels, from `POST .../labelling-queue`, while its lease holds. The outcome then counts as a queue label, weighted by its band in the fit. pattern: ^lq_[0-9A-Z]{26}_[0-9]+$ examples: - lq_01JAYQ3K5V9W2X4Y6Z8A0B1C2D_0 Error: type: object required: - error properties: error: type: object required: - code - message properties: code: $ref: '#/components/schemas/ErrorCode' message: type: string details: type: object description: >- Code-specific detail, such as the scan estimate on a refused query. Every `internal` error has `request_id`. properties: reason: type: string description: > Why, where one code has more than one cause. On `forbidden`, `payment_overdue`: a payment has been unpaid for 14 days, so the organization's keys are read-only until it is paid; reads still work, and paying restores writes within a few minutes. A key that is read-only itself is `forbidden` without a `reason`. On the events feed's `invalid_request`, `cursor_expired`. suggested_context: description: >- On a create without a context recipe, the recipe suggested from the namespace's documents, or null when there are none. oneOf: - $ref: '#/components/schemas/SuggestedContext' - type: 'null' DocumentId: type: string description: >- Up to 128 bytes. Never exactly `.` or `..`, which a URL path can't carry. pattern: ^(?!\.\.?$)[A-Za-z0-9._:/-]+$ minLength: 1 maxLength: 128 Name: type: string description: >- A judgment, attribute or threshold name. Names are path segments in field references, so they never contain `.`. pattern: ^[A-Za-z0-9_-]+$ minLength: 1 maxLength: 128 Timestamp: type: string format: date-time description: RFC 3339, UTC. ErrorCode: type: string description: | HTTP status by code: `invalid_request` 400, `unauthorized` 401, `budget_exceeded` 402, `plan_required` 402, `forbidden` 403, `not_found` 404, `conflict` 409, `too_large` 413, `engine_version_unavailable` 422, `insufficient_labels` 422, `idempotency_key_reused` 422, `rate_limited` 429, `internal` 500, `engine_unavailable` 503, `unavailable` 503. enum: - invalid_request - unauthorized - forbidden - not_found - conflict - too_large - rate_limited - budget_exceeded - plan_required - engine_unavailable - engine_version_unavailable - unavailable - insufficient_labels - idempotency_key_reused - internal SuggestedContext: type: object description: | The recipe suggested for a create without `context`, in the refusal's `details.suggested_context`. From a sample of the documents the definition applies to: it keeps short text, numbers, booleans and small attributes, and leaves out ids, timestamps, links, binary-looking and very long values, and rarely present fields; `excluded` says why for each. `max_tokens` is set to keep the context in the standard size class, at most 2,000. The same documents always give the same recipe. required: - recipe - cost_per_answer - whole_state_cost_per_answer - sampled_documents - excluded properties: recipe: $ref: '#/components/schemas/ContextRecipe' cost_per_answer: type: - number - 'null' minimum: 0 description: >- Mean judgments per answer, counted by size class, over the sample with this recipe and the definition's question. The suggested `max_tokens` is at most 2,000, the standard ceiling, less the question's tokens, so a context that fills it stays standard with its question. whole_state_cost_per_answer: type: - number - 'null' minimum: 0 description: The same with the whole `state`. sampled_documents: type: integer minimum: 1 excluded: type: array description: >- Each path left out, and why, so you can put back what the question needs. items: type: object required: - path - reason properties: path: type: string reason: type: string enum: - id - timestamp - url - binary - long - rare - too_many ContextRecipe: type: object description: | What the engine sees. A create without it is refused with a suggested recipe, unless `use_context` asks for the suggestion or the whole `state`; a version with no recipe sends the whole `state`, subject to the engine limit. additionalProperties: false properties: fields: type: array description: Paths included verbatim, in order. items: type: string pattern: ^(state|attributes)(\.[^.]+)+$ last_n: type: object description: Per array path, keep the last n elements. additionalProperties: type: integer minimum: 1 window: type: object description: >- Per array path whose elements have a timestamp field `at`, keep elements within the window. additionalProperties: $ref: '#/components/schemas/Duration' max_tokens: type: integer minimum: 1 description: Hard cap; the compiler truncates from the end of the last field. related: type: object description: > Up to 4 relations, by name: documents in the same namespace that point at the judged one, or the one document it points at. Each renders as a `related.` entry after `fields`, and a write that changes what it renders makes the judged document's answer `pending` until it is judged again (for a referenced document, only inside the re-judge scope; outside it, the answer is `stale` with `stale_reason` `referenced_changed`). minProperties: 1 maxProperties: 4 propertyNames: $ref: '#/components/schemas/Name' additionalProperties: $ref: '#/components/schemas/Relation' previous: $ref: '#/components/schemas/PreviousRecipe' Duration: type: string description: A whole number and a unit (`s`, `m`, `h` or `d`), such as `7d` or `1h`. pattern: ^[1-9][0-9]*[smhd]$ Relation: type: object description: | Documents that point at the judged document. They are ordered newest `created_at` first: `window` keeps those created within the window, then `last_n` keeps the newest n. At least one of `last_n` and `window` is required, and at least one of `fields` and `aggregate`. A relation reads at most the newest 1,000 documents. With `join: {theirs: "id", mine: "attributes."}` the relation reads the one document the judged document points at, its referenced document, and takes neither `last_n` nor `window`. With `join: {theirs: "attributes.", mine: "attributes."}` it is a blocking relation over the documents that share the judged document's key value. It needs `window`, takes `last_n` up to 254 (up to the `options.from` candidate limit when a `choice` chooses among it with `options.from`), and a change to its block re-judges the block's judged documents. `fields`, `aggregate` and `match` may read plain judgments' answers (`answers..`), banded or cut. additionalProperties: false required: - match - join allOf: - description: A relation renders fields or aggregates. if: not: required: - fields then: required: - aggregate - description: >- A relation over the documents that point at the judged one is bounded by `last_n` or `window`. if: properties: join: type: object properties: mine: const: id then: anyOf: - required: - last_n - required: - window - description: >- A relation that reads the referenced document takes neither `last_n` nor `window`. if: properties: join: type: object properties: theirs: const: id then: not: anyOf: - required: - last_n - required: - window - description: A blocking relation needs `window`, and reads at most 254 documents. if: properties: join: $ref: '#/components/schemas/BlockingJoin' then: required: - window properties: last_n: type: integer maximum: 254 properties: match: $ref: '#/components/schemas/RelationMatch' description: Which documents the relation reads. join: $ref: '#/components/schemas/RelationJoin' last_n: type: integer minimum: 1 maximum: 1000 description: >- Keep the newest n. Not on a relation that reads a referenced document; at most 254 on a blocking relation, and at most the `options.from` candidate limit (see Limits) when `options.from` chooses among it. window: $ref: '#/components/schemas/Duration' description: >- Keep documents created within this long before the later of the judged document's newest write and its newest related creation, at the answer's watermark. An edit to an old related document never moves the window. Not on a relation that reads a referenced document. fields: type: array description: >- Paths to include from each document, rendered under `records`, newest first. An entry may be a `BandedField`, which renders a number as its band. An entry may also be a plain judgment's `answers..value` and `answers..thresholds.` as they are, and its `.p`, `.score` and `.dist.