Fernwick Air: promoting a model on one intent

workflow.yaml is AWML 0.6, as it was pushed. The registry draws, checks and fits it converted to AWML 0.8.

awml: "0.6"
title: "Fernwick Air: promoting a model on one intent"
description: |
  A candidate model sits the intent's held-out exam, the judge grades it, and code prices its
  mistakes with the confusion matrix against the people's. If it beats them, finance, legal and
  compliance, CS ops, QA and data science, and platform each say yes to the exact change before the
  policy row changes. Fernwick Air is fictional.
scopes:
  - { id: tokenised, label: "tokenised: Fernwick staff", processing: { anthropic: any } }
  - { id: ops, label: "ops: Fernwick staff", processing: none, in: tokenised }
parts:
  - { id: put_up, kind: trigger, label: "Someone puts a candidate model up for an intent and tier", wait: P7D }
  - { id: gold, kind: context, label: "The scored set: real cases, each with its true intent and agreed answers, a slice held out for exams", in: tokenised, untrusted: true, ref: chrismdp/counting-house-gold#gold }
  - { id: policy_table, kind: context, label: "The policy table: each intent's model, tier, SOP, tools and approvers", in: tokenised }
  - { id: candidate, kind: ai, op: draft, label: "The candidate answers each held-out case, as the clerk would", in: tokenised }
  - { id: exam, kind: context, label: "The candidate's exam answers", in: tokenised }
  - { id: judge_trust, kind: context, label: "The intents the judge is trusted on", in: tokenised, ref: chrismdp/counting-house-judge-test#judge_trust }
  - { id: judge, kind: ai, op: check, label: "The judge grades the candidate's answers against the agreed ones", in: tokenised }
  - { id: costs, kind: context, label: "Finance's cost for each confusion-matrix cell", in: tokenised }
  - { id: scores, kind: context, label: "Every contact's scores, by intent and contact reason", in: tokenised, ref: chrismdp/counting-house-ongoing#scores }
  - { id: price, kind: code, op: check, label: "On an intent the judge is trusted on, count the candidate's mistakes by confusion cell, price them, and compare with the people's", in: tokenised }
  - { id: propose, kind: code, op: do, label: "Ask to change the intent's policy row", in: tokenised }
  - { id: finance, kind: review, label: "Finance's yes to the exact change: costs and approval thresholds", in: tokenised }
  - { id: legal, kind: review, label: "Legal and compliance's yes: may this intent reach this tier", in: tokenised }
  - { id: csops, kind: review, label: "CS ops' yes: the SOP and the floor are ready", in: tokenised }
  - { id: qa, kind: review, label: "QA and data science's yes: the eval and the judge", in: tokenised }
  - { id: platform, kind: review, label: "Platform's yes: the tools, gateway and orchestrator", in: tokenised }
  - { id: change, kind: outside, label: "Change the intent's policy row: Ask", in: tokenised }
lines:
  - { from: put_up, to: candidate, kind: start }
  - { from: gold, to: candidate, kind: data, label: the held-out cases }
  - { from: policy_table, to: candidate, kind: data, label: the intent's SOP }
  - { from: candidate, to: exam, kind: data }
  - { from: candidate, to: judge, kind: start }
  - { from: exam, to: judge, kind: data }
  - { from: gold, to: judge, kind: data, label: the agreed answers }
  - { from: judge, to: price, kind: start, label: its grades }
  - { from: judge_trust, to: price, kind: data }
  - { from: costs, to: price, kind: data }
  - { from: gold, to: price, kind: data, label: the people's own mistakes }
  - { from: scores, to: price, kind: data, label: the intent's live and shadow scores }
  - { from: price, to: propose, kind: start, label: "[cheaper than the people]" }
  - { from: propose, to: finance, kind: start }
  - { from: finance, to: legal, kind: start, label: "[yes]" }
  - { from: legal, to: csops, kind: start, label: "[yes]" }
  - { from: csops, to: qa, kind: start, label: "[yes]" }
  - { from: qa, to: platform, kind: start, label: "[yes]" }
  - { from: platform, to: change, kind: data, label: "[yes]" }
  - { from: change, to: policy_table, kind: data }