Skip to main content
Some outcomes arrive after the answer: an invoice goes overdue some weeks after Brussle judges it. Give the judgment a horizon, and each result you record counts against the answer it followed, once per window, so frequent judging never makes the numbers look better. The engine answers well about what a document shows now, and near chance about what will happen. For a question about what will happen, ask the engine a narrow question about the document as it is, keep counts from related documents as features, and let the fit on your outcomes weigh them. The features page shows the full pattern.

Set a horizon

horizon is part of the judgment’s definition: the time after each answer in which an outcome counts against that answer. It is a whole number and a unit (s, m, h or d), such as "30d". The default is 0s, for labelled examples. A change to it makes a new version. Then post the outcome when the event happens:
When the event already reaches you as a write, such as the invoice’s attributes.status becoming overdue, an outcome rule posts it for you. A judgment with a horizon has no labelling queue: its outcome is about the time after the answer, which nobody can label from the document today. It gets positives as they happen, and negatives as you post them or from implicit negatives. It also cannot be backtested.

How outcomes meet answers

Brussle matches outcomes to answers with outcome windows. (A window here is an outcome window, not a relation’s window.) For example, a judgment has a 30-day horizon, and Brussle judges a document on day 0. That answer’s window runs to day 30. An outcome “it happened”, observed on day 12, labels the answer made on day 0.
  • Windows. A document’s first answer opens a window, which runs for the horizon from the time of the answer. Answers made while it is open do not open windows. The first answer after it closes opens the next. Each window is measured on the answer that opened it.
  • Labels. An outcome labels the window that its observed_at falls in. Two outcomes in one window count once. An outcome you posted wins over one a rule made. Otherwise a true wins.
  • “It did not happen”, after the window. A false observed after a window closed, and before the next one opens, labels the window that closed. Post “it did not happen” on day 40 and it counts for the answer made on day 0. A true already in that window stays, unless a rule made it.
  • Late outcomes count. Brussle matches outcomes each time the fit or the report reads them, so a result you post weeks after the event labels its window at the next fit.
  • Engine epochs. A window counts for the epoch of the answer that opened it.

Outcomes that count nowhere

The report counts these in unmatched_outcomes:
  • an outcome observed before the document’s first answer (before_first_evaluation);
  • an event (true) observed after a window closed and before the next answer (after_horizon).

After the event

Once Brussle sees a true for a document, answers made from then on have nothing left to measure: the event already happened, and often the write that recorded it also caused the answer. Their windows count nowhere, not as a false and not labelled by an outcome. An outcome in such a window counts in unmatched_outcomes.after_positive. For an event that can happen again, such as a second refund, only the first counts.

Deleted and rewritten documents

  • A document written again after a delete starts a new window at its first new answer, even after an event. An answer is about one life of its document, so an outcome observed after the first answer of the new life never labels a window from before the delete.
  • An outcome observed while its document is deleted labels no window, and counts in unmatched_outcomes.no_evaluation. A document remembers the last 8 deletions of its id, so this holds however late you post the outcome, as long as the deletion is still recorded when Brussle stores or reads the outcome, or when you write the id again.

Post what did not happen

The correction needs the invoices that did not go overdue too. Do one of these:
  • Post false after the window has passed, at any time. It answers the window that closed, unless an event was already posted in it.
  • Turn on implicit negatives, if you record every event. Then each window that closes with no outcome counts as false.

Turn on implicit negatives

On a bool judgment with a horizon that is not zero, you can post only what happened, and let every other window count as “no”. This example also records each event from your writes with a rule:
Send it with PATCH /namespaces/{ns}/judgments/{name}. It is a setting, like thresholds, so it makes no version. The next fit and every report use it. Each PATCH replaces the whole outcomes object, so send the rules with it. With implicit negatives on:
  • A window that closes with no outcome counts as false, for the answer that opened it.
  • A window that is still open counts nothing yet. An answer from yesterday on a 30-day judgment does not count, because its outcome can still come.
  • Outcomes you post, or that rules make, always win in their window. Post a positive quickly, or let a rule make it from the write that records the event. Until then, the window counts as a negative when it closes.
  • A window whose document was deleted before it closed is left out, unless an outcome labels it: the event may have happened to a deleted document, so its silence is not a “no”. After the deletion is no longer recorded, the document counts as existing until its newest answer.
  • A window opened after the event is left out. The event already happened to that document.
  • The report shows the counts: implicit outcomes in each epoch’s implicit_negatives and in outcomes_by_source, and the windows left out in censored_predictions: window_open (not closed yet), document_deleted (deleted before it closed) and after_positive (opened after a true).
  • A judgment with many judged documents is fitted on a representative sample of them, with the outcomes posted for those documents, so the share of positives stays true.
When not to use it. Turn it on only when a missing outcome really means “it did not happen”. Leave it off when:
  • you record only some of the events, such as refunds from one payment system but not another;
  • outcomes arrive long after the horizon, such as chargebacks that take 90 days to report on a 30-day judgment;
  • the judgment has the default 0s horizon. Brussle refuses implicit negatives there.
A namespace can inherit a judgment from a template. Set implicit negatives on the template’s prefix path. Every namespace under it shares the setting, and the pooled correction.