Scoring method decisions, October 18, 2026
Status: Decision record for Hugo's review. Not committed, not published, not applied to any profile. Prepared by Claude Fable 5.1 from Hugo Campos's stated decisions in a working session on October 18, 2026, and revised the same day after GPT-6 Astra's independent cold review (project record, not published) and Hugo's adjudication of it. The v1.3 rubric and worksheet implementing these decisions were written and revised the same day. No public score, profile or deployment changed.
Why this record exists
The v1.2 rubric had been a proposal since September 7, 2026. Its adoption gate (section 10) required Hugo's review of the anchors and a two-reviewer coding study. The study became the agency comparison pilot (project record, not published), which Hugo closed on September 7 before the independent rating sheets were completed. No record of a decision on the anchors was written.
Since then, six reviews cited a "pause on assigning new numbers" as the reason for publishing without an axis position (Ari, aidoctorscan, Chronilogix, Anistratenco, CareClinic, Avie). That pause appears only in text inside the rubric draft itself. It was never recorded as a project decision in STATUS.md, an audit or any other decision record. Meanwhile 49 entries carry provisional numbers assigned by narrative placement, as described in the score method audit (project record, not published). AskEllyn received 80 on September 7 from public sources plus one session. Anistratenco and CareClinic, with the same evidence type three weeks later, received no number. Newer reviews were held to a stricter standard than older ones.
This record ends the undocumented pause, records the decisions needed to adopt a provisional editorial method, and names what remains open.
Decisions
D1. The v1.2 control anchors are adopted as the provisional control measure
The ordinal anchors 0, 20, 40, 50, 60, 80, 90 and 100, with their deciding conditions in section 5A, are adopted as written. No weights, no interpolation, no deductions for unknowns. Unknown conditions produce a candidate set or "unresolved," never a number. A candidate set is shown as discrete possible positions, never as a range, and no midpoint is derived.
Reason: the anchors make a judgment reproducible after evidence has been coded, which the legacy numbers never were. The spacing is editorial and is labeled as such.
D2. Transparency is an editorial eligibility standard for the control axis
A tool receives no control position unless its service operator and model developer are named publicly on the product's own site or terms, and the tool names its operator accurately when asked directly inside the assessed function. A tool that fails is listed, receives an agency posture and a prose control classification (including an adverse one), and carries a specific flag: operator not disclosed, model provider not disclosed, operator disclosure inaccurate, or disclosure unresolved. It receives no position and no candidate set. Section 3A defines the evidence states.
Reason: Hugo's stated position is that a tool that will not say who runs it cannot be judged on informed terms from inside it. The gate is stated as an editorial publication standard, not as a claim that there is nothing to measure. Astra's review (item 1) noted that an opaque tool with evidenced external control would otherwise vanish from the axis while a transparent one stays visible. The prose control finding preserves that evidence. Hugo kept the gate and accepted the rewording.
D3. Only tools the patient participates in directly receive a control position
Direct participation means the patient takes part in the assessed AI interaction, personally or through assistance they authorize (a care partner typing at their direction, assistive input, an authorized proxy). Receiving an output produced in a clinician-operated workflow is not participation. Tools that fail this condition remain in the directory with an agency posture, a prose control finding where the evidence supports one, and prominent refusal, correction and binding-remedy findings, labeled "control axis not applicable: clinician-operated."
Reason: this is a scope choice for the numerical axis. It does not assert that patients lack authority over tools they never operate. A patient can hold a binding remedy over a clinician's AI output. That authority is recorded in the supporting measures rather than on the axis. (Astra item 6, accepted; item 5, accepted: "operates directly" replaced by direct participation with authorized assistance.)
Affected entries today: Abridge (43), Nabla (40), Dragon Copilot (36), OpenEvidence (18), nVoq Voice (28), BastionGPT (16). Their numbers move to historicalAxisPosition. OpenEvidence leaves the homepage hero-axis examples.
D4. A single position requires protocol-complete interaction
A single position may be published only when the assessed function has been tested in two fresh sessions using the common protocol (P1 to P6, COMMON_TESTS.md), each including the self-disclosure prompt, with the packet frozen and hashed, and the deciding conditions resolve. Documentary and partial-interaction reviews publish a candidate set or "unresolved." Where documentary evidence alone would resolve to one position, the profile says "position withheld pending required testing" and does not widen the set.
Reason: the rubric already states that public-source-only reviews cannot claim the behavior checks passed. The legacy numbers were assigned without those checks. Applying the rule to old and new profiles alike removes the double standard.
Consequence: only AskEllyn and ChatGPT currently have protocol evidence in the repository. Every other legacy number is a pre-method editorial placement and will be labeled as such until re-derived.
D5. The control axis covers functions with a patient-set purpose
The assessed function must be one in which the patient can state a purpose for the interaction and the tool can be redirected and questioned within it. Conversational assistants, record-grounded question answering and drafting tools qualify. Live interpretation, record connectors and fixed-question intake do not. They are assessed for agency effect and the supporting measures with no control position, labeled "control axis not applicable: no patient-set purpose."
Reason: Hugo's question on October 18 was whether an interpreter is worth scoring at all. It affects agency (language access, correction, refusal, data reuse) and so stays in the directory with a posture. It has no purpose the patient sets, so the control axis has nothing to place. This rule replaces the idea of per-function interaction protocols: the common protocol applies to every tool on the axis, and tools it cannot test are off the axis for a principled reason. (Astra item 8, resolved by scope rather than by new protocols.)
Affected entry today: No Barrier (43) moves to historicalAxisPosition. CareClinic's connector is off the axis; a CareClinic-plus-chosen-host configuration can be a review object under section 2A.
D6. The agency posture leads
Every profile and listing shows the section 7 posture first (expands, constrains, mixed, potentially agency-expanding, unclear). The control position follows, as evidence for how the posture arose. Evidence coverage and review status are shown separately from both.
Reason: Hugo's statement of the project's purpose on October 18: the question is whether a tool expands or constrains agency, liberation from oppressive institutional algorithms. The number is how constraint usually happens, not the finding. Posture first also resolves the AskEllyn problem: a tool can be at 90 and still fail to liberate, and the card should say "Mixed, and here is why" rather than "90, with a caveat." This is a presentation change and will require renderer work before publication.
D7. "Expands agency" requires publication authorization
Expands agency may be published only with Hugo Campos's authorization, recorded by name and date, in addition to section 7 rule 3's evidence requirement. This is a publication rule, not an evidence tier.
Reason: it is the strongest claim the site makes about a tool. Astra (item 9) was right that a signature is not evidence, so the rule is named for what it is. Hugo expressed no preference. Fable made the call. Hugo can overrule.
D8. Human check when coders agree
When both model coders agree, Hugo still confirms the frozen boundary (section 2A) and the transparency gate states (section 3A) before publication. Codes stay with the coders. Disagreements and unresolved gates come to Hugo as before.
Reason: two coders can share an omission. The boundary and the gates are where a shared error would do the most damage. Astra item 13, accepted in this limited form. Hugo expressed no preference. Fable made the call. Hugo can overrule.
Resolved items
- O1 (constraint types). Option A adopted and confirmed by Hugo. Testimony substitution and deferral displacement are material constraints with step-by-step tests in section 7, drawn from Astra items 10 and 11. The earlier claim that v1.2 excluded them is corrected: v1.2 did not exclude them, it lacked tests for them (Astra item 12).
- O3 (rendering candidate sets). Discrete positions, visually distinct, sorted separately or by a disclosed rule. No range, no midpoint (Astra item 14). Not yet built.
- O4 (testing capacity). The procedure is now in rubric section 10 and
AUTOMATION.md, with D8's agreement check added.
Open items
- O2. Scenario families. Addition, October 30, 2026: record-grounded assistants (portal assistants answering from the patient's own chart, such as Emmie) cannot be protocol-tested with fictional information. Candidate routes, not decided: a deployer test account with a synthetic chart, or sessions run by a consenting patient on their own record, redacted before the packet is frozen (boundary note EM11). The common protocol uses one fictional breast-cancer scenario. Tools on the axis that serve other conditions or tasks (appeals, condition coaching, records interpretation) need a scenario variant with the same P1 to P6 structure, versioned and frozen before use. Not yet drafted. Under D5 these are disease and task variants of the one conversational protocol, not per-function protocols.
- O5. Boundary cases on paper. Done October 30, 2026. Results, rule-note dispositions and fixes F1 to F27 are in research/v1-3-reviews/BOUNDARY-CASES-2026-10-30.md (project record, not published). Two items need Hugo (H1, H2 below). Before publication, apply the revised rules to K Health PatientGPT, Epic Emmie (noting that Sutter runs a separate billing assistant, per Astra item 7), a CareClinic-plus-host configuration, and No Barrier as the first excluded case. Record the result as a calibration file. This is the test of the rules before any live profile uses them.
- O6. Renderer. Built locally October 30, 2026, unpublished:
scripts/hugoscore_method.py(rules),scripts/method_render.pyandsite/app.js(display),scripts/compile_v13_reviews.py,scripts/build_v13_preview.py. Inert on current data. Posture-first cards and listings, candidate sets as discrete positions, legacy labels, control-axis-not-applicable labels. Not yet built. - O6a. Agency gauge design. Settled October 30, 2026: three groups (see below). Hugo asked for a gauge with constraint on the left and expansion on the right. A single needle would reintroduce a net score and would show Mixed as neutral, so the planned design is two independent arcs with no needle and no center number: the left arc fills per documented material constraint, the right per demonstrated patient ability (understand, ask better questions, act), solid when shown in testing and hatched when documented only, with the posture word in the center and the control position as text beneath. Mockup at
research/rubric-v1-3-draft/agency-gauge-mockup.svg. Pending Hugo's choice: group the red arc into three segments (refuse or leave, correct or challenge, steered or deceived) or show every named constraint type separately.
- O7. Public framework page. Redrafted October 30, 2026 at
research/rubric-v1-3-draft/framework-page-draft.md, pending Astra's second read. To be redrafted in the register of Hugo's Ashby Village page, agency questions first, who-directs second, what-we-check third, with each plain statement linked to the rubric section it summarizes so the public page cannot produce a different judgment from the rubric (Astra item 15).
Disposition of GPT-6 Astra's review
| # | Astra's point | Disposition |
|---|---|---|
| 1 | Withholding the number for opacity conflates publication rule with measurement | Accepted as rewording. Gate kept as editorial eligibility standard. Prose control finding preserved. (D2) |
| 2 | Gate lacks states for missing or conflicting evidence | Accepted. Seven evidence states and explicit eligibility rules in 3A. |
| 3 | Naming an operator is not identifying the correct one | Accepted. Role record in 2A, accuracy check and "inaccurate" state in 3A. |
| 4 | Single public-operator rule does not fit composed or patient-run workflows | Accepted. 2A handles composed workflows and private deployments. |
| 5 | "Operates directly" confuses physical interaction with authorship | Accepted. Direct participation with authorized assistance, worked examples. (D3) |
| 6 | Patients can have authority over systems they do not operate | Accepted as rewording of D3's reason. Scope choice, not a claim. |
| 7 | Unit of assessment vulnerable to convenient boundaries | Accepted. Frozen boundary record in 2A. Four cases become O5. Sutter billing/medical split noted. |
| 8 | Mandatory chat protocol penalizes non-conversational tools | Resolved by scope (D5) rather than per-function protocols. Non-conversational functions are off the axis. |
| 9 | Evidence tiers not exhaustive, mixed with review status | Accepted. Coverage and review status separated, every permitted posture listed, documentary singleton rule, Expands authorization named as a publication rule. (D7) |
| 10 | Testimony substitution needs statement-level test | Accepted nearly as written. |
| 11 | Deferral displacement needs a test for what was displaced | Accepted nearly as written. |
| 12 | Posture rules let a modest benefit dominate a serious constraint; O1 overstated | Accepted. "Offsets" removed, same-function requirement, coexistence not balance, O1 corrected. |
| 13 | Independent coding needs an explicit claim and conflict record | Accepted in part. Procedure written into section 10. Agreement check limited to boundary and gates. (D8) |
| 14 | Candidate sets should not become ranked ranges | Accepted. (D1, O3) |
| 15 | Public framework draft changed rules | Accepted. Page to be redrafted with section links. (O7) |
Settled October 30, 2026
Hugo Campos answered the two items that were pending, in a working session with Claude Opus 5.5 on October 30, 2026. Both answers followed the recommendation put to him.
- D7 symmetry: adopted. "Constrains agency" now also requires Hugo Campos's publication authorization, recorded by name and date, in addition to section 7 rule 2's evidence requirement. Reason: it is the site's strongest adverse claim about a vendor, and treating both ends alike guards against a charge of bias. Until authorization is recorded, a profile whose evidence meets rule 2 is published as Unclear with the constraint stated in full, and the profile says the Constrains finding awaits authorization. Rubric sections 7 and 8 are updated.
- O6a gauge grouping: three groups. The red arc has three fixed segments mirroring the green arc: refuse or leave, correct or challenge, steered or deceived. A segment fills when any material constraint in that group is supported. Every named constraint is still listed beneath the gauge with its evidence. Reason: a segment per constraint would turn arc length into a count, so three findings would look three times worse than one, which reintroduces the net score the gauge was designed to avoid. The constraint-to-group mapping is in rubric section 7.
Answered October 30, 2026
- H1. Settled October 30, 2026. Hugo: institutional versus patient-directed is a spectrum, not a binary. The ultimate patient-directed end is a patient running, training or distilling their own model locally with their own data, which is unlikely to exist. The other end is an institutional AI that is patient-facing but strict about what it allows the patient to do. The more guardrails steer the patient toward an institutional goal, the more institutional it is. Applied: the control line is that spectrum, the rails are coded through C1 to C4 and the steering constraint, who deploys a tool is evidence and not the answer, and there is no fixed binary label on cards (rubric section 3).
- H2. Revised by Hugo, October 30, 2026, 12:52: coders are Claude Opus 5.5 and GPT-6 Astra only. This replaces the earlier answer below. The conflict rule is unchanged. Consequences: entries that name neither company get both coders; entries that name one company get one coder, the other, and stay AI drafts labeled "one eligible coder"; entries that name both companies get no v1.3 review and keep their legacy label with the reason stated (BastionGPT, CureWise, Fight Health Insurance, My Doctor Friend, Noah, Wysa on current evidence). Earlier answer, superseded: add a third company. Section 10 needs two coders from different vendors, neither from the tool's vendor nor its model developer. With Claude and GPT-6 Astra only, tools built on Claude or GPT models have one eligible coder. Hugo chose to add a third vendor's model (for example Google Gemini) as the second coder for those tools. Until it is added, those profiles state "one eligible coder" and remain AI drafts.
Disposition of GPT-6 Astra's second read, October 30, 2026
The review is at research/rubric-v1-3-draft/review-gpt-6-astra-second-read-2026-10-30.md (project record, not published). Astra found 9 of its 15 October 18 items resolved and 6 partly resolved, one fix (F4) that relaxed D2, eight public-page discrepancies and eight code defects. Claude Opus 5.5 applied every item the same day. None required a new decision from Hugo.
| Item | Disposition |
|---|---|
| B1. F4 relaxed D2 | Accepted. T3 passes only on a named service operator. Another role-record participant is not found. T1 requires each joint operator. |
| B2. F1 fallback precedence | Accepted. An incomplete search blocks rules 3 and 4, not an established Mixed or Constrains. |
| A2, K1. Gate states | Accepted. Valid states per test, T3 substitute when not applicable, inaccessible never counts as support. |
| A4, C5. Private and composed deployments | Accepted. Gate rules for private patient-run deployments, each joint operator named, model provider defined on the page. |
| A7. O5 K Health object | Accepted. The calibration says plainly it worked the consumer chat. A PatientGPT case remains. |
| A9, C1. Expands coverage | Accepted. Rule 3 requires protocol-complete coverage, matching section 8. The page says so. Potentially's meaning on the page corrected. |
| K2. Unknown directory scope | Accepted. Any U scope gate gives "directory scope unresolved." |
| K3. Authorizer | Accepted. Only Hugo Campos authorizes. |
| K4. Unauthorized Expands | Accepted. Shown as Unclear with the evidence in full, symmetric with Constrains under D7. |
| K5, C7. Solid segments | Accepted. Solid for a HugoScore protocol observation or independent operational study, labeled which. Documents alone never solid. |
| K6, C8. Coder rules | Accepted. Same-vendor coders rejected, open conflict checks reported, "one coder so far" shown on profiles and stated on the page. |
| K7. Ungrouped findings | Accepted. Listed as text and flagged for method revision, not rejected. |
| K8. Constraint search inputs | Accepted. Every search element must be explicit. Contradictory completeness codes rejected. |
| C2, C3, C4, C6. Page sentences | Accepted. Page rewritten to carry every outcome-changing condition. |
Publication order
- Hugo reads this record and the revised rubric. Done when he says so.
- The public framework page is redrafted (O7).
- GPT-6 Astra re-reads only the revised sections (1, 2, 2A, 3A, 7, 8, 10) and the page draft, and records remaining disagreements. Hugo settles them. Request prepared October 30, 2026:
research/rubric-v1-3-draft/second-read-request-2026-10-30.md. Order changed: the boundary cases (step 4) were run first, so Astra reads the calibrated text. - The four boundary cases are worked on paper (O5).
- The criteria are published on hugoscore.org before any tool is scored under them. Authorized by Hugo Campos on October 30, 2026 ("publish phase 1"): method pages and framework page only.
- All 60 profiles are labeled by method version, evidence coverage and review status. Legacy numbers display as "legacy editorial placement, pre-method." No re-scoring in this step.
- The seven entries outside the control axis (D3 and D5) are relabeled.
- The six paused entries and the next new submission are reviewed under the method.
- Legacy entries on the axis are re-derived in batches, blind to the old number, old numbers moved to history, with a public changelog entry per batch.
Corrections to existing documents
SCORING_RUBRIC_DRAFT.mdsection 10 andhugoscore-anchor-calibration-2026-09-07.mddescribed AskEllyn's live position as 75. The public profile has been 80 since a second reviewer restored it on September 7. Both now carry a dated note.SCORING_RUBRIC_DRAFT.mdsection 10 referred to "the pause on assigning new numbers." This record is the first and only decision about that pause and ends it.CLAUDE.mdandSTATUS.mdstate that control-axis scope is narrower than directory scope.- The Epic Emmie profile treats Sutter's billing and medical assistants as one. Queue a refresh.
What this record does not do
It does not validate the anchors as a measurement instrument. It does not change any published number. It does not establish that two coders will read evidence the same way. It records Hugo's editorial decisions so that old and new profiles can be held to one visible standard.