Repository audit — 2026-08-05

A stress test of the framework against its own stated vision, run at commit 32a6c12. Six independent audits covered the closed-loop layer, interface parity, adoption and upgrade, failure and observability, security and safety-as-implemented, and claims-versus-reality with test quality.

This page records gaps the development backlog does not already track. The backlog remains the curated work list; this is the evidence behind additions to it.

Findings are not deleted when they are fixed. The evidence and the reasoning are the durable part, and a finding that recurs is more informative than one that vanishes.

Resolution log

Date	Closed	Note
2026-08-06	G1	Stale documentation figures. Reopened and closed twice; see the finding.
2026-08-06	E2, E4, C-series disclosure, G3	Compatibility policy written, the adapter guide carried through to a working installation, and the statements that outran the code corrected.
2026-08-06	A1, A2, C5, D6	The runtime no longer replays an action whose outcome it recorded as unknown, an invalid result no longer repeats the action that produced it, the declared lifecycle is enforced on every state write, and the untested failure branches have tests confirmed to fail first.
2026-08-06	D1, D2, D3, D5	The Blender rebuild runs in CI and a skip is now a failure; the propagation graph is wired into `make lint`, covers every tracked file, reports a nonexistent path, and is right about `packages/capabilities`; the policy suite went from one test to fourteen and catches six mutations that all slipped past the original; the boundary checker fails on an unmapped package; the SDK is exercised against the real application.
2026-08-06	B2, B3, B5, B8, B9, C1, C2, C6, C7, F1, F2, F4, F5, F6, F7, C8 (API), A4	The campaign can no longer claim an environment its laboratory did not declare, survives a failed run, records why it stopped, and its runs are reachable from it. Reads no longer write, a manifest can no longer bind arbitrary code unchecked, policy evidence is verifiable, and generated laboratories no longer leak `.env.*`. The CLI reports one line with a meaningful exit code, `validate` resolves what it certifies, the API distinguishes its failures and stops echoing adapter text, and the tool catalogue is real.
2026-08-06	C3, E1, F3, B1 (read side)	Manifests name credentials with `${env:NAME}`, fail closed, and never print a resolved value back. Alembic is the only schema writer, a pre-Alembic store is adopted rather than rejected, and a schema-comparison test caught 23 indexes the initial migration never created. A declared timeout now binds a blocking adapter — measured 2.01 s to 0.12 s — and a blocking adapter no longer stalls concurrent runs. Campaigns are readable from the CLI, API, SDK, MCP and context pack, projected from their own events.
2026-08-06	A5, A2 (root cause), D7 (additive half), E1 follow-on	One `retry_safety` declaration now answers both whether to repeat a dispatch and what a timeout may claim, so `SAFETY.md`'s "retry safety" is expressible for the first time. A `deny` rule an earlier `allow` fully covers is refused at load without changing evaluation. A store behind a destructive revision is refused on open, and each revision's declaration is verified by applying it and diffing rather than believed.
2026-08-06	B6	The optimizer contract can express batch proposal, multiple objectives, declared constraints, a validated search space, uncertainty and acquisition provenance, without adding a required method. A decision is recorded before the run it causes and names the evidence it rested on. Batch execution is real and bounded by a laboratory-declared parallelism; the runner still does not schedule around a resource lease, which a test states rather than a comment.
2026-08-06	B4, B7	The optimizer contract lives in `opensdl-core`, so a third-party optimizer depends on a protocol and a few frozen models rather than on storage, policy, workflows and SQLAlchemy; the sole boundary exception is gone, and `CampaignDefinition` can declare what the runner accepts, checked in both directions. A campaign resumes from its own record: completed work is not repeated, iteration numbering continues, a second start under one identifier is refused, and a resume over a run whose physical outcome is unknown is refused rather than laundered.
2026-08-10	A4 (blast radius), third instance of the A1/A2 class	Restart reconciliation asked "may this be repeated when nothing reported an outcome?" for itself instead of reading `retry_safety`, contradicting the timeout path in the same file. One helper, `may_repeat_without_outcome`, now serves every path that asks it without an outcome in hand. An interrupted task whose capability declares `repeatable` is recorded `FAILED` and resumes; every other declaration, and an unknown capability, still records `INTERVENTION_REQUIRED`. So a restart no longer permanently ends a campaign built from capabilities that had declared repeating harmless. A4's blast radius is bounded accordingly, and the `--help`, docstring, CLI reference and onboarding text that described the old unconditional behaviour are corrected. The other half is unchanged: no operation acknowledges an intervention, so `INTERVENTION_REQUIRED` remains terminal for everything else — still tracked in Backlog §4.
2026-08-10	A7	The consecutive-failure limit counts batches that completed nothing rather than observations that did not succeed, so a laboratory asked to run more at once than it owns no longer stops on its first batch reporting the failure as systematic. A batch that completed nothing still counts every attempt in it, so a genuinely failing laboratory stops as promptly as before, and `batch_size=1` is unchanged in every case. A6 remains open, and so does the larger half of A7: a candidate that lost a race is still recorded as a failed evaluation it never received.
2026-08-10	A3 (restated, one limit closed)	This finding was written the day before `OutcomeConstraint` landed and had read as more absolute than the code ever since. A deployment-declared feasibility criterion with the recorded third outcome does exist at campaign level; what is missing is narrower and is now stated as four specific limits. The sharpest of them — that a criterion which is not a measurement had no expression at all — is closed by `equals`. The remaining three are one output path per constraint, no access to task inputs or history, and no per-task postcondition, which is the form `SAFETY.md:34` requires.
2026-08-07	Decision 4 (the C5 mutation case), B7 (remaining half)	`RunCreated` carries a canonical digest of the workflow document it captured and a resume must present the same document, so a non-terminal run can no longer be turned into a different execution under the original operator's name; `supersedes` mints a new run naming the one it replaces, recorded on both. Starting a run is one conditional write over the states the declared machine already permitted, so two callers can no longer both enter `run_workflow` on the same running run. `CampaignObservation`, `Suggestion` and `CampaignProblem` are typed models with generated schemas, and the hand-rolled serialisation whose keys matched nothing published is gone.

Decisions the owner has to make

Four findings are small to implement and cannot be implemented without deciding what the system should do. Each is stated with what is true today, the options, and a recommendation. Decisions 2, 3 and 4 have since been taken and are marked below. Only decision 1 remains open: its additive half is implemented, and the part that would change how policy evaluates is deliberately left for an announced release.

1. Should an explicit `DENY` override an earlier `ALLOW`? (D7)

Today: rules are sorted by ascending priority, the first match wins, and evaluation stops. Deny does not override allow. An operator who adds deny: sim.hazardous at priority 20 to a configuration already carrying allow: "*" at priority 10 gets no error, no warning, and no effect. The rule is dead code. Every comparable system does the opposite: XACML has deny-overrides, IAM gives explicit deny precedence, RBAC has no deny at all.

Option	Consequence
Leave it, documented	Cheapest. The trap remains, and the documentation is the only thing standing between an operator and a deny rule that does nothing. Already asserted by a test named for the trap.
Reject a shadowed `DENY` at load time	Small. Turns silence into a startup error naming both rules. Does not change evaluation, so no existing configuration changes meaning. Cannot catch a partial shadow — a deny narrower than the allow above it in some dimensions but not all.
Evaluate every matching rule, `DENY` wins	Matches every comparable system and removes the class. It silently changes the meaning of existing configurations — any deployment relying on an early allow shadowing a later deny flips behaviour, and it flips toward refusing, which is the safe direction but is still a change nobody asked for. A compatibility event under the policy written this session.

Recommendation: reject a shadowed DENY at load time now, and schedule deny-overrides for the next minor with a changelog entry. The load-time check gets the operator out of the trap immediately without changing what any running laboratory does, and it is honest about being partial. The semantics change is right, but it should arrive announced rather than inside a patch that says it is fixing test coverage.

2. Should a timed-out task be recorded as failed, or as ambiguous? (A5)

Today: a timeout records FAILED. But asyncio.wait_for cancels the adapter coroutine — it does not stop the instrument, which may well have received the command. The physical outcome is exactly as unknown as in the cancellation branch, which correctly records INTERVENTION_REQUIRED with the error "physical outcome is unknown".

This now interacts with a fix that has already shipped. Resume refuses to re-dispatch a task in an ambiguous state, and FAILED is deliberately resumable so a failed run can be retried. So the one path that under-reports its own uncertainty is also the one path that will silently re-dispatch.

Option	Consequence
Leave it	A timed-out physical action can be repeated by a resume. Contradicts the reasoning behind the A1 fix.
Record `INTERVENTION_REQUIRED`	Consistent with cancellation and with what the runtime actually knows. Every timeout then needs a human before that run continues, which for a simulator-only profile is friction with no benefit — and today every shipped adapter is a simulator. Contradicts the documented timeout behaviour and an existing test.
Let the capability declare it	A capability whose action is idempotent or has no physical effect can be safely retried after a timeout; one that dispenses cannot. This is the same missing field `SAFETY.md:41` asks for under "retry safety", which A2 also ran into.

Recommendation: the third, and treat it as one piece of work with A2's root cause. Both findings are the same gap — the runtime has no way to know whether repeating an action is safe, so it has to guess, and it currently guesses differently in two places.

Decided and implemented 2026-08-06. CapabilityDefinition.retry_safety carries three values, not two, because safe to repeat and never repeat cannot hold the common real case — a dispense that failed to connect versus one abandoned mid-pour. The third, repeatable_if_not_dispatched, needs evidence or it collapses into the others, so it is unlocked only by an adapter raising NotDispatchedError. Nothing verifies that claim; it is the adapter's statement about its own transport, and a timeout is explicitly never one, because the runtime stopping its wait establishes nothing about the equipment.

The default is not_repeatable — the safe reading rather than the compatible one. The asymmetry decides it: defaulting permissive costs an unrevisited physical capability a repeated dispense, while defaulting strict costs it some automatic retries and turns some timeouts into interventions. Measured blast radius of the default alone was four test capabilities and nothing else in the suite. SAFETY.md:41's "retry safety" is expressible for the first time — though expressible is not enforced: the conformance harness still does not require an explicit declaration, which is left open deliberately rather than flipping a public harness for every third-party adapter.

3. Should opening a laboratory for writing migrate it? (E1 follow-on)

Today: Database.initialize() runs the migration history whenever a laboratory is opened for writing, which is what keeps every existing laboratory working and is what create_all() effectively did before. The consequence is that a future destructive migration will run during an ordinary opensdl run, with nobody asked and no backup taken.

Option	Consequence
Migrate on open, as now	Nothing breaks, no laboratory needs attention after an upgrade, and one day a migration that drops a column runs inside a campaign.
Refuse a store behind head, require `opensdl migrate`	Explicit, backup-able, and the operator chooses the moment. Breaks every existing laboratory until somebody runs a command, including automated ones.
Migrate additive revisions on open, refuse destructive ones	Keeps the common case silent and stops only where stopping matters. Requires each revision to declare which it is, and a declaration nothing verifies is the failure mode this register is full of.

Recommendation: the third, with the declaration checked rather than trusted — a test can compare each revision's operations against its declared kind, which is the same shape as the schema-comparison test that caught the index drift.

Implemented 2026-08-06, and the work falsified part of this recommendation. I wrote above that "the risk is theoretical, because no destructive migration has ever been written". That was wrong. Revision 0002 drops schema_versions, and the verification caught it immediately: the declaration check reports {"table:schema_versions"} against a revision I had assumed was additive.

It matters more than a miscount, because every laboratory applies 0002 — including one created today, since 0001 creates that table and 0002 removes it. So "refuse destructive revisions on open" cannot be a blanket rule without refusing every laboratory that exists. The refusal therefore carries a named GRANDFATHERED_DESTRUCTIVE set holding 0002, with the reason written beside it, so the exemption is reviewable rather than inferred from age and adding to it is a visible decision.

The verification does not read the declaration and believe it. It upgrades a store to the revision's predecessor, reflects the schema, applies the revision alone, and diffs against the reflected before-schema — so a table appearing in the diff means the revision dropped it. A separate scan flags raw SQL in upgrade(), because a DELETE empties a column and leaves the schema byte-identical, which no schema comparison can see.

4. What identifies a resumable run? (C5 follow-on)

Today: resubmission against a non-terminal run is checked only by workflow id. A FAILED or INTERVENTION_REQUIRED run can be resumed with a different step list, and the new steps are attributed to the original operator while the workflow of record stays the original. Two callers can also enter concurrently on the same running run when none of its tasks is active.

Option	Consequence
Digest the workflow into `RunCreated` and require a match	Closes the mutation cleanly and gives every run a verifiable record of what it was asked to do. Rejects a legitimate case: fixing a broken step and resuming.
Forbid only a changed step list, allow parameter edits	Narrower. Harder to specify, and "the same steps" is not obviously the property that matters.
Leave it, and require a new run	Simplest and arguably correct: a run is a record of an execution, and changing what it executes makes it a different execution.

Recommendation: the first. A run that cannot prove what it was asked to do is not evidence, and this framework's whole claim is evidence. Resuming with a repaired workflow should mint a new run that references the old one, which is also what makes the provenance readable afterwards.

Taken, 2026-08-07. The digest is over the document RunCreated already embedded, using the same canonical JSON the twin binding uses — now shared as opensdl_core.canonical_digest rather than spelled out twice. A run recorded before the field existed has its digest recomputed from that embedded document, so nothing stored becomes unresumable. The referencing mechanism is supersedes, on run_workflow, POST /runs and opensdl run. The concurrent-entry hazard closed with it: RUN_TRANSITIONS never permitted running to running, but the validator returns early when the current state equals the target — right for an idempotent write, wrong for claiming a run — so claiming is now a separate conditional write at the store.

Found while fixing, not by the audit

Three defects surfaced during remediation that the audit missed. They are recorded here because the pattern matters: a finding that lists three instances of a class is a sample, not an inventory.

PRAGMA foreign_keys is never enabled, while packages/storage/src/opensdl_storage/db_models.py:43 declares ondelete="CASCADE". SQLite ignores foreign keys unless the pragma is set per connection; PostgreSQL enforces them always. So the single place the two supported backends provably diverge is untested in both directions, and cascade behaviour differs by backend today. S.
The human-task adapter stamps "attested": True unconditionally (adapters/human-task/src/opensdl_adapter_human_task/adapter.py:92), including when outcome is not_completed or deviated. An exported provenance bundle therefore carries an attestation of work that was explicitly reported as not done. This is worse than C-series' original point about the word being unverified. S.
CapabilityDefinition.version has no readers and a workflow step cannot request a version. A capability's schema can change under a workflow already bound to it with nothing to detect it. M.
The declared lifecycle was wrong in four places, and two of them were contradicted by tests that already passed. RETRYING → SUCCEEDED was absent, so the machine said a retried attempt could never succeed while test_retry_then_success_... demonstrated the opposite; RETRYING → INTERVENTION_REQUIRED was absent while test_recovery_marks_ambiguous_tasks... produced it. Enforcing the machine (C5) was what surfaced this — a specification nothing consults does not stay correct, and this one had been wrong since it was written. That is the same class as D1 and D4: green, and constraining nothing. Resolved with C5.
D8 — nothing surfaces a red build, and nothing stops one being built on. S to disclose, M to enforce. CI was red for three consecutive commits before anyone noticed, and the cause was a test that had never passed in CI: it asserted an option appears in Rich-rendered --help, and the hosted runner renders narrower than a developer shell, so it truncated. It passed locally every time. Three separate things had to be absent for that to run as long as it did. main reports as protected but carries no protection rule, so no status check is required to merge and nothing refuses a push over a failing build. ci.yml had no workflow_dispatch, so a gate that stopped reporting could not be re-run without pushing another commit — which is how a later infrastructure stall became indistinguishable from a code failure. And nothing notifies: a red build is discovered by someone thinking to look. Two of the three are fixed; requiring a green check before merge is a repository setting rather than a code change, and it is the one that would have caught this on the first commit rather than the fourth. The narrower lesson is the familiar one in a new place — an assertion against rendered output tests the renderer, and its failure arrives detached from the change that caused it.
A5 — a timed-out task is recorded FAILED, which claims more than the runtime knows. S asyncio.wait_for cancels the adapter coroutine (engine.py:341-352), so the instrument may well have received the command. The physical outcome is exactly as unknown as in the cancellation branch, which correctly records INTERVENTION_REQUIRED and says "physical outcome is unknown" — but the timeout branch records a clean failure, and under the new A1 rule a FAILED task is resumable. So the one path that under-reports its own uncertainty is also the one path that will silently re-dispatch. Changing it contradicts the documented timeout behaviour and an existing test, so it needs a decision rather than a patch.
D7 — an explicit DENY rule can be silently dead, and this is a design bug rather than a documentation gap. M. Policy is first-match-wins ordered by ascending priority, and deny does not override allow. So an operator who adds deny: sim.hazardous at priority 20 to a configuration that already carries a broad allow: "*" at priority 10 gets no error, no warning and no effect — the rule is dead code. Every comparable system does the opposite: XACML has deny-overrides, IAM gives explicit deny precedence, and RBAC has no deny at all. The behaviour is now documented and asserted, including a test named for the trap, but the shipped semantics remain the surprising ones. Two remedies, smallest first: reject at load time a DENY fully shadowed by an earlier ALLOW; or evaluate every matching rule and let DENY win. Related and now also covered by a test: risk_classes is the only selector that compares strings rather than globbing, so risk_classes: ["R*"] reads as a wildcard, matches nothing, and silently authorizes zero capabilities.
C5 closed the terminal case; the mutation case is open. Resubmission against a non-terminal run is still checked only by workflow id (engine.py:85-97), so a FAILED or INTERVENTION_REQUIRED run can be resumed with a different step list, attributed to the original operator. Two callers can also still enter run_workflow concurrently on the same RUNNING run when none of its tasks is active. M.

How to read this

Every finding carries a file and line so it can be re-verified rather than re-argued. Sizes are S (under a day), M (a few days), L (a week or more, with contract design).

Findings marked [verified] were confirmed directly against the code by the synthesising author, not accepted on an auditor's report. Findings without that marker are single-source and should be re-confirmed before work starts.

One reported finding was refuted; it is recorded below rather than dropped, because knowing a check was made is worth as much as the finding would have been.

Refuted

"make test fails on every clean clone; CI has almost certainly been red." It has not. CI run 30966397037 executed the overlay step on all three interpreters and reported 15 passed, 1 skipped each time, and make test returns 0 locally. The proposed mechanism — that the pytest console script resolves from the project virtualenv and misses the --with-editable overlay — does not hold either: uv run --locked --with-editable ./examples/digital-twin-surrogate/adapter python -c "import opensdl_adapter_cell_surrogate" resolves to the editable source tree.

The 1 skipped in that output is finding D1 below. The same evidence that refuted this claim confirmed a real one.

A. Physical-safety-adjacent defects

Nothing here is a live hazard: every shipped adapter is a simulator and SAFETY.md is scrupulous about that. All four are far cheaper to fix before an operational adapter exists than after.

A1 — Resume re-dispatches actions whose physical outcome is recorded as unknown [verified] · S

packages/runtime/src/opensdl_runtime/engine.py:99-103 rebuilds step_outputs from SUCCEEDED tasks only. A task left INTERVENTION_REQUIRED by restart reconciliation or cancellation therefore falls into pending, is passed to _execute_step as existing_task, and is set straight to RUNNING at engine.py:279.

The error string that task carries is "physical outcome is unknown" (engine.py:313-315,374-375). The system records that it does not know whether the physical action occurred, and then repeats it. docs/architecture/agent-native-operation.md:183 requires the opposite: "Ambiguous acknowledgement creates an intervention and no automatic replay."

The guard is a few lines — refuse to resume a task that is neither SUCCEEDED nor PENDING. Backlog §4 tracks intervention acknowledgement as a missing capability; this is an active defect in shipped code and should be tracked separately.

A2 — A data-validation failure re-dispatches the physical action [verified] · M

validate_instance(result.output, definition.output_schema, ...) sits inside the retried try at engine.py:289-295. A schema violation is caught by the generic except Exception at engine.py:331 and the retry loop calls adapter.execute again.

So a dispense that reported an out-of-range volume is dispensed again. SAFETY.md:41 requires adapters to define "retry safety"; the runtime has no field in which to declare it and no way to honour it. CapabilityDefinition (packages/core/src/opensdl_core/models.py:74-90) has no idempotency or retry-safety attribute.

A3 — No acceptance criteria, data-quality gate, or postcondition verification anywhere · L

SAFETY.md:34 requires postcondition verification as a distinct record. docs/architecture/agent-native-operation.md:116 defers it with no backlog item. The only mechanism that exists is JSON Schema validation of result.output, which is adapter-declared (not deployment-declared), has no access to the task inputs, and has no access to history — so it cannot express "measured mass within 5% of commanded mass", and cannot detect drift.

The reference showcase makes the gap concrete: its own contract declares "quality": {"type": "string", "enum": ["ok"]} (examples/digital-twin-surrogate/adapter/src/opensdl_adapter_cell_surrogate/adapter.py:215). The vocabulary cannot express a bad measurement. An instrument reporting degraded would fail schema validation, be physically re-measured (A2), and then fail the run.

There is no path from "the instrument says this datum is untrustworthy" to "record it, quarantine it, escalate". This is the failure mode that costs a 200-run campaign silently; everything else announces itself.

Partly addressed at the campaign level on 2026-08-06, one day after this was written, and this finding did not say so. OutcomeConstraint (packages/core/src/opensdl_core/campaign.py:257, added in b654008) is deployment-declared, is evaluated against a run's outputs, and carries exactly the third outcome asked for above: a violating observation is recorded, keeps its numbers, is excluded from the best and from reaching a target, and does not fail the run. So the sentence "the only mechanism that exists is JSON Schema validation" is no longer true.

Four limits were identified. The first is now closed; three remain, and they are the residual work:

~~Numeric only.~~ Closed 2026-08-10. as_number (campaign.py) rejects bool on the correct ground that a bool is an int in Python and is not a measurement, and nothing had been put beside it, so "the solver reported convergence: yes or no" was inexpressible. equals now states an exact value, keeping bool, int and str apart because True == 1 in Python. Floats are excluded deliberately: exact equality on a measured quantity is almost never the criterion, and lower with upper says the intended thing.
One output per constraint. output is a single dotted path, so a criterion relating two quantities has to be precomputed by whatever parses the code's output, which moves the criterion into the adapter and out of the document.
Outputs only. There is no access to the task's inputs or to prior runs, so "within 5% of what was commanded" and any drift criterion remain unexpressible — the original point above, unchanged.
Campaign level only. A workflow task has no postcondition gate, which is the form SAFETY.md:34 actually requires.

A4 — `opensdl doctor` silently reconciles active runs [verified] · M

packages/cli/src/opensdl_cli/main.py:88-90 calls system.start(), which calls runtime.recover_incomplete_runs() (apps/controller/src/opensdl_controller/system.py:169-174). That transitions every RUNNING run to INTERVENTION_REQUIRED, releases its leases, and appends events (engine.py:360-396).

opensdl doctor --help says, in full: "Check database, artifact store, and adapter health." A seeded running run became intervention_required after doctor, which exited 0 printing "passed": true and mentioned nothing.

Running a health check during a live campaign destroys the operational record of the experiment in flight and reports success. The same happens on API startup (apps/api/src/opensdl_api/app.py:59). Only .agents/skills/orient-lab/SKILL.md step 6 warns of this — nothing in --help, the reference docs, or the code.

A6 — A timed-out task releases its instrument while saying the equipment may still be moving · M

Found 2026-08-10, while fixing the lease bound. Not addressed, because the fix depends on a mechanism that does not exist.

The timeout path records, in its own words, that the runtime "stopped waiting; it did not stop the equipment, and nothing reported an outcome, so the physical outcome is unknown", moves the task to INTERVENTION_REQUIRED, and states that "a person must establish what the equipment did before this run continues". The finally around the retry loop then calls release_leases(task.id) unconditionally, so the instrument that may still be moving is immediately available to the next task that asks for it.

The reasoning and the resource behaviour contradict each other: the record says a person must establish what the equipment is doing, and the lease says anyone may now command it.

The obvious fix — hold the lease until the intervention is acknowledged — needs an acknowledgement operation, and there is none; INTERVENTION_REQUIRED is terminal for everything, which Backlog §4 already tracks. Holding the lease with no way to release it would strand the instrument until the lease expired, which is a different wrong answer rather than a better one. So this is recorded and left, and it should be settled together with intervention acknowledgement rather than before it.

A7 — Losing a resource race stops the campaign and blames the science · M · resolved

Found and reproduced 2026-08-10; resolved the same day. The stop rule now counts batches that completed nothing rather than observations that did not succeed.

A campaign records a lease failure as a failed iteration, which is deliberate and tested — "a lease failure is a laboratory fact and not a framework error". What is not deliberate is what that failure then does. Every non-succeeding observation increments consecutive_failures, and reaching max_consecutive_failures (default 3) stops the campaign with FAILURE_LIMIT and the detail "the failure is systematic rather than routine".

A laboratory that declares more parallelism than it has exclusive instruments produces a loser in every batch. Four candidates, max_parallel_runs=4, one exclusive instrument, defaults otherwise:

stop_reason : failure_limit
stop_detail : 3 consecutive iterations failed, so the failure is systematic rather than routine;
              last error: resources busy: ['probe-bench']
successes   : 1 [{'x': 1.0}]
candidates never evaluated: [{'x': 2.0}, {'x': 3.0}, {'x': 4.0}]

The campaign stops on its first batch having measured one of eight candidates. Nothing was wrong with the instrument, the chemistry, or the seven candidates it never evaluated; the laboratory was asked to run four things at once and can run one. Unattended operation is the point of the loop, and this ends it on the first batch while naming a cause an operator will go looking for in the wrong place.

Two consequences beyond the early stop:

A candidate is recorded as a failed evaluation it never received. _ESTABLISHED_RUN_STATES includes FAILED, so a resume reads the outcome off that run rather than re-dispatching, and a parameter point that lost a race is never measured.
The optimizer is told the candidate failed. It is evidence about the laboratory's schedule, offered as evidence about the parameter point.

Resolved by the second option. "Systematic" now means a batch in which nothing completed. A batch that got a candidate through is a working laboratory, so the candidates that lost a race in it no longer count toward the limit. A batch that completed nothing still counts every attempt in it, so a genuinely failing laboratory stops exactly as promptly as before — three failed attempts, whether they arrive one batch at a time or all at once — and a campaign with batch_size=1, the default, behaves identically to before in every case. _trailing_failures reads the same way, so a resume cannot inherit a limit the running loop had already reset.

Not counting contention at all was rejected: a resource held by something outside the campaign would leave the loop running forever with nothing to show. The two larger options remain open and are the honest end state rather than this one — re-queuing the loser instead of recording a failed evaluation, and scheduling around the lease so the contention does not arise. Backlog §4 tracks the scheduling half.

Still true, and not addressed here: a candidate that lost a race is recorded as a failed evaluation it never received, _ESTABLISHED_RUN_STATES includes FAILED so a resume never re-dispatches it, and the optimizer is still handed that observation as evidence about the parameter point. Fixing those is the re-queue option above.

B. The closed loop is not a contract

The framework's headline claim rests on this layer, and it is the least first-class subsystem in the repository. It also explains why the digital twin is structurally blind to the decide half of the loop — same root cause, different symptom.

B1 — Campaigns are unreachable from every interface [verified] · L

campaign appears in zero of the CLI, HTTP API, SDK, MCP transport, OperatorGateway, and ContextPack. It exists only in packages/runtime/src/opensdl_runtime/campaign.py, reachable by writing bespoke asyncio Python, as examples/simulated-color-mixing/run_campaign.py demonstrates.

Consequences: a campaign cannot be started, observed, or stopped by an operator, an agent, or a remote client; the API server process cannot host one; and an agent calling lab.describe cannot see that its laboratory is mid-campaign. The twin command group — an Unreleased feature — has more interface coverage than the headline v0.1 feature.

Backlog §3 aligns surfaces that already exist. Campaign is absent from all four, so there is nothing to align.

B2 — The campaign path bypasses the manifest environment [verified] · S

CampaignRunner.run hardcodes environment: str = "simulation" and operator_id: str = "software/campaign" as default parameters (packages/runtime/src/opensdl_runtime/campaign.py:47-48). The controller's own submission path uses environment=self.manifest.spec.environment (apps/controller/src/opensdl_controller/system.py:217), and policy is evaluated against that environment (engine.py:217).

examples/simulated-color-mixing/run_campaign.py:24 constructs the runner directly and does not pass environment, so the default is what ships.

A laboratory that sets environment: production and writes policy permitting only simulation gets its direct submissions denied and its campaign — the one unattended path — executed. The persisted runs then record environment="simulation", so the provenance record is false about where the work happened.

B3 — One failed run aborts the campaign, and the optimizer never learns from failure · M

campaign.py:61-96 has no exception handling around run_workflow (line 68) or the score extraction (line 71). A single failure propagates out: no CampaignCompleted is emitted, the CampaignResult is lost along with every successful iteration, observe() is never called, and the failed candidate never enters history — so a re-launch re-suggests it forever.

There is no CampaignObservation representation for "attempted, invalid" or "attempted, failed". In a real laboratory a clogged tip or an off-scale reading is routine, and it is information.

B4 — No campaign resume; no optimizer state contract; re-running duplicates physical work · L

run() always starts with an empty history (campaign.py:52) and range(max_iterations) (campaign.py:61); nothing reads prior DecisionRecorded payloads back. The Optimizer Protocol (campaign.py:21-23) has no state() / load_state(). recover_incomplete_runs has no campaign analogue.

Re-running with the same campaign_id emits a second CampaignStarted with no idempotency check and restarts iteration numbering at 0, so (campaign_id, iteration) is not unique and decision provenance becomes ambiguous. A fitted surrogate, trust region, or RNG state is unrecoverable.

The framework has restart reconciliation for the thirty-second thing and none for the three-week thing.

B5 — Run and task events carry no `campaign_id` · S

ReferenceRuntime._emit accepts campaign_id (engine.py:405) and no caller supplies it. Measured on the example database: 91 of 1108 events are campaign-scoped, and those are only the three campaign event types. RunRow has no campaign_id column; there is no campaigns table and no decisions table.

list_events(campaign_id=X) returns no execution history. After a 200-run campaign there is no query that returns what it did. This is a schema gap, so it gets more expensive with every run recorded.

B6 — The optimizer protocol cannot express Bayesian optimization · M–L

suggest(history) -> dict | None and observe(obs) -> None (campaign.py:21-23) with CampaignObservation(iteration, candidate, score: float, run_id, outputs).

Missing: batch/parallel suggestion (so a lab with eight reactors runs them one at a time, and q-EI is unrepresentable); multi-objective and Pareto; constraints; a declared parameter space the framework can validate a candidate against before leasing resources; uncertainty — notable because Quantity.uncertainty and Observation.uncertainty exist elsewhere in core; and acquisition provenance. suggest is also synchronous, so a GP refit blocks the event loop including asyncio.wait_for timeout enforcement for concurrent runs.

Decision rationale is a template string, f"optimizer selected candidate for iteration {iteration}" (campaign.py:85), and the decision is recorded after the run it selected, with evidence_run_ids naming the run it caused rather than the runs it was based on. For a framework whose thesis is evidence preservation, the decision record contains no decision information.

B7 — `CampaignDefinition` is a dead contract, and the split costs real things · M

CampaignDefinition lives in packages/core and generates packages/schemas/jsonschema/campaign.schema.json; it is imported by nothing that executes. CampaignRunner.run takes nine keyword arguments duplicating its fields and does not accept it. CampaignObservation is a plain frozen dataclass in runtime with no schema and no validation, serialised by a hand-rolled _observation_json (campaign.py:124-133) whose camelCase keys match no published schema. This violates the repository rule that public models are exported as versioned schemas.

Concrete cost: because the optimizer contract lives in runtime, every optimizer plugin must depend on the whole execution stack. scripts/check-boundaries.py:55 encodes this as the sole exception in the boundary map. A third party publishing a BoTorch optimizer pulls in storage, policy, workflows and SQLAlchemy to import a five-field dataclass.

B8 — Hardcoded `sample_id` breaks computational campaigns · S

campaign.py:67 does inputs.setdefault("sample_id", ...) — a domain-specific input name baked into the domain-neutral runtime with no way to disable it. The framework's own generated template teaches additionalProperties: false, so a compute-only workflow following that pattern is rejected outright. The "computational laboratories" half of the project's own tagline cannot run through CampaignRunner.

B9 — Only `max_iterations` controls the loop · M

No target threshold, no convergence or stagnation detection, no wall-clock budget, no reagent or cost budget, no maximum-consecutive-failures rule. CampaignCompleted does not record why the campaign stopped, so budget-exhausted and converged are indistinguishable in the log. There is no external stop, pause, hold or abort, and no CampaignAborted or CampaignFailed event type.

C. Security posture versus documentation

SECURITY.md §"Secure defaults" is written in the present tense and describes the target, not the system. Four of its nine bullets have no implementation. The safety documentation, by contrast, matches reality closely and is unusually honest.

C1 — Generated laboratories will commit `.env.production` [verified] · S

Root .gitignore:14-16 has .env, .env.*, !.env.example. The generator template packages/cli/src/opensdl_cli/templates/laboratory/.gitignore.j2:6 has only .env.

Since environment variables are the only sanctioned credential channel (C3) and multi-manifest workflows are what the docs recommend, this is the most likely real secret leak in the project. Two lines.

C2 — `CapabilityBinding.config` is accepted and silently ignored [verified] · S

apps/controller/src/opensdl_controller/system.py reads adapter_config.config and pack_config.config but never binding.config (packages/schemas/src/opensdl_schemas/manifest.py:38-42).

An operator writing operating limits into a capability binding — exactly where you would look for them — gets no error and no effect. SAFETY.md:23 requires the deployment to deny requests outside the validated operating domain; there is no mechanism, and the field that looks like one does nothing. Silently ignored safety configuration is worse than an absent field.

C3 — No secret mechanism exists, and the docs describe one that does not · M

docs/concepts/manifests.md:5 states credentials are supplied "through environment variables or a secret provider". There is no ${ENV} interpolation in load_manifest (packages/schemas/src/opensdl_schemas/manifest.py:97-102), no secret-provider interface, no SecretStr, and no secret scanning in make lint or CI. Adapter config is verbatim plaintext from manifest to adapter.

An operator integrating a real instrument today has exactly one option: paste the credential into opensdl.yaml, the file the docs designate as belonging in Git. The words "secret" and "credential" appear nowhere in the backlog.

C4 — `operator_id` is caller-asserted and is the policy subject · S (disclose) / L (fix)

Every interface takes the actor identity as a caller-supplied parameter and passes it to policy.evaluate (engine.py:217) and into every event's actor_id (engine.py:411). The operators: dimension of PolicyRule therefore provides no assurance, and every provenance record's attribution is unverified self-declaration.

Backlog §8 tracks authentication. What is missing is any note that a documented policy dimension is decorative until then — an operator writing a per-operator deny rule gets silent failure.

C5 — Lifecycle transitions are never enforced; run resubmission corrupts provenance [verified] · M

validate_run_transition and validate_task_transition (packages/core/src/opensdl_core/lifecycle.py:49-60) have zero production callers — they are only re-exported from __init__.py. Repositories.update_run writes any state over any state.

Reachable over the unauthenticated API today: POST /runs accepts a caller-supplied run_id; engine.py:85-97 checks only workflow.id, not the step list, emits no new RunCreated if one exists, and forces state to RUNNING. New steps then execute and are attributed to the original operator, while the run's workflow of record stays the original. Run outputs are overwritten.

Policy is still evaluated per step, so this is not a policy bypass — it is provenance forgery plus actor misattribution. SECURITY.md:37 names event tampering a high-priority class.

C6 — Plugin allowlist exists but only inside a skill helper · S

system.py:83 loads manifest-declared plugins — arbitrary installed code — with no allowlist and no provenance check. The good protection, validate_reference_adapter_plugins (packages/capabilities/src/opensdl_capabilities/plugins.py:88-122), has no call sites outside .agents/skills/develop-workflow/run.sh and its unit test.

In a laboratory repository where an agent edits the manifest, this is the shortest path from "agent writes a file" to "arbitrary code runs in the process that talks to equipment". No user-facing doc says plugin: is a code-execution boundary. Backlog §8's signed-promotion item is the heavyweight future version; calling the validator that already exists from from_manifest is small.

C7 — Policy evidence is unverifiable · S

PolicyDecision.policy_version carries a free-form string from the manifest into the durable PolicyEvaluated event. Nothing binds it to the rule content, so every rule can change while the version stays constant and the recorded evidence is identical. The project already does this correctly for the twin, which pins a canonical-JSON digest and refuses a mismatch.

C8 — Other security items not on the backlog · S each

Unbounded caller-supplied JSON Schema compiled and executed on the event loop (engine.py:67-68) — a pattern keyword with catastrophic backtracking blocks the server; no bound on step count or /events?limit. Error strings and workflow inputs flow verbatim into unauthenticated responses and the permanent event log (app.py:123,145; engine.py:287,337), which becomes an exfiltration path once a real adapter raises with a connection string. Event rows have no hash chain or append-only constraint despite SECURITY.md:50 promising immutability. No SAST, secret scanning, or dependency audit in CI; ruff has flake8-bandit off. Dependabot covers uv and github-actions only — the viewer's npm dependencies and both Docker base images are unmonitored. The generated-lab CI template abandons the repository's own pinning discipline: floating action tags and uv sync without --locked.

D. Checks that certify nothing

The repository's own history teaches that a passing check can be worthless. These are the places where that is currently true.

D1 — The Blender reproducibility test always skips in CI [verified] · M

Blender appears zero times across all four workflow files, so examples/digital-twin-surrogate/tests/test_scene_reproducibility.py:47-60 takes its pytest.skip path. CI run 30966397037 shows 15 passed, 1 skipped on every interpreter.

This is the strongest correctness check the repository owns and the one VALIDATION.md leans on hardest. The claim is true — it passes locally with Blender installed — but it is unenforced. The remaining asset tests verify the GLB against reports produced by the same build, so loosening a threshold in check_scene.py and regenerating would pass everything. A Blender point release also silently disables it, and nothing surfaces the skip.

No workflow, Make target, or validator invokes opensdl propagate or loads propagation.yaml.

opensdl propagate packages/nonexistent/foo.py returns {"affected_nodes": []} with exit 0 — a path that does not exist is indistinguishable from one with no impact. Measured against git ls-files, 93 of 455 tracked files (20.4%) match no node, including all of .agents/skills/, all of scripts/, Makefile, AGENTS.md, ROADMAP.md, VALIDATION.md and CHANGELOG.md. For a repository whose thesis is agent-native operation, the skills tree being invisible to the blast-radius tool is the sharpest version of this.

It is also factually wrong about a real dependency: packages/capabilities/** is filed only under the conformance node, with no edge to runtime or the operator interfaces, though the runtime, controller and CLI all import it directly.

This has already cost the repository — see G1.

D3 — Policy has 100% coverage and one test · M

packages/policy/tests/test_policy.py is 26 lines with one assertion pair. It reads 100% statement and branch coverage only because PolicyRule.matches() is a single return joining four and-ed conditions.

Never tested: risk_classes matching with a non-* value; operators matching with a non-* pattern; rule priority ordering (never exercised with two rules, so precedence between a broad allow and a narrow deny is unverified); an explicit DENY rule — the one denial test relies on default_effect falling through, so the deny branch never returns; and rule_id propagation.

Policy is the authorization boundary, and this is the clearest instance in the repository of a coverage number certifying nothing. Related: precedence is undocumented — lower priority number wins, first match wins, and deny does not override allow.

D4 — Conformance lets a trivially wrong adapter pass · M

packages/capabilities/src/opensdl_capabilities/conformance.py:17-120 validates each case's inputs and outputs against the adapter's own schemas, using the adapter's own cases. It grades its own exam. A sim.measure_mass returning a constant passes every check.

Nothing checks typed errors, retries, abort, invalid-input rejection, timeout, or determinism — though adapters/AGENTS.md requires the first three and "deterministic" is a headline claim. The harness's own failure branches are untested: if a bug made run_adapter_conformance return passed=True unconditionally, no test in the repository would notice.

D5 — Smaller cases · S each

scripts/check-boundaries.py:79 silently skips any package absent from its map rather than failing, so a new package ships boundary-free; the surrogate example adapter is currently unchecked. The SDK test suite stubs the client's own _request, so OpenSDLClient is never exercised against the FastAPI app and renaming an API route passes every test. packages/cli/tests/test_scaffold.py:56 guards its assertion behind if adapter.is_symlink(), so the mirror path asserts nothing. Two tautologies in packages/operators/tests/test_tools.py:7 and packages/sdk/tests/test_client.py:9. 231 of 237 tests carry no marker, so -m integration runs 2 tests and misses test_api.py and test_migrations.py entirely.

D6 — Untested failure branches that matter · S each

ResourceBusyError has zero tests on either branch (engine.py:248-261) — resource leasing is the only concurrency-safety mechanism in the framework. Permanent non-timeout adapter failure after retries is untested (engine.py:352). Adapter output-schema rejection is untested (engine.py:291-295), so the runtime's only defence against a lying adapter is unverified. The resume path (engine.py:85-95,111) has no test.

E. Adoption and upgrade

E1 — An existing laboratory's database can never be upgraded · M

Database.initialize() calls create_all() and hand-stamps "0001" (packages/storage/src/opensdl_storage/database.py:40-47), bypassing Alembic entirely, on every CLI command that touches a lab. create_all is CREATE TABLE IF NOT EXISTS — it will never add a column or backfill. The stamped version is never read for comparison and is not Alembic's alembic_version table.

The documented recovery command fails against any laboratory that has ever run (table schema_versions already exists), and no alembic stamp instruction exists anywhere. There is no opensdl migrate command; Alembic is a runtime dependency unreachable from the CLI. The generated laboratory has no database/ directory, no alembic.ini, and no migration docs.

Real drift already exists: db_models.py declares index=True on 23 columns and 0001_initial.py creates zero indexes. README.md:12 advertises "database migrations" as shipped and VALIDATION.md marks them Verified.

E2 — No compatibility, versioning, or deprecation policy · S (write) / M (enforce)

Three sentences exist repository-wide, all deferring the question. No VERSIONING.md, COMPATIBILITY.md, UPGRADING.md, or DEPRECATION.md; zero occurrences of "breaking change"; no DeprecationWarning anywhere; no cross-version test suite. ROADMAP.md:89-91 makes 1.0 conditional on "the compatibility suite has external adopters" — no compatibility suite exists.

scripts/check-version.py enforces string equality of one version across 22 packages. It never parses the version — no PEP 440 check, no monotonicity, no relation to the changelog or tags. It detects a typo, not a breaking change.

Both apiVersion surfaces are hard Literal pins with extra="forbid", so any additive future field breaks older readers and there is no version dispatch or converter. Generated JSON Schemas carry no $id, $schema, or version field, and the drift check is byte-for-byte — removing a required field passes CI as long as the schemas were regenerated.

E3 — Distribution failure modes the backlog does not name · S each

Backlog §1 correctly targets publication. It does not name what bites today: the generated uv.lock records the wheelhouse as a registry (source = { registry = "../opensdl/dist" }, ×22), uv.lock is not in the generated .gitignore, so users will commit it, and a colleague cloning that repository cannot resolve at all — with no documented recovery. Generated dependency floors have no upper bound, so a future breaking release satisfies every generated laboratory. release.yml is workflow_dispatch-only with contents: read, no Trusted Publishing and no tagging, and git tag -l is empty despite the changelog claiming a released 0.1.0a0.

E4 — The first extension step dead-ends · S

opensdl adapter create produces a good package and nothing installs it — LookupError: unknown adapter plugin. The fix (a path dependency plus a [tool.uv.sources] entry) is documented nowhere. docs/guides/add-adapter.md is ten lines and stops at generation. There is also a location conflict: the README example writes to ../my-lab/adapters, the generated AGENTS.md says src/my_lab/adapters/.

E5 — `opensdl init` prints only a path · S

No next steps, no warning that packages are unpublished, no git init — though every document insists the laboratory should live in its own repository. Three lines of output would absorb most of the confusion in E3. Relatedly, the generated laboratory's simulator is stateful — sim.measure_mass requires a prior sim.mix_color — and nothing says so, so the first workflow a user writes themselves fails.

F. Interfaces and diagnostics

F1 — `opensdl validate` certifies broken configurations · M

validate returns Manifest valid and exit 0 for a manifest naming a nonexistent adapter plugin, and Workflow valid for a workflow referencing an unknown capability. Both fail immediately at composition or dispatch. doctor does not catch it either. This is the command every document puts in front of a new user. Backlog §4 frames dry-run validation as a new feature; the existing false positive should be tracked as a defect.

F2 — `GET /tools` advertises a contract nothing implements · S

packages/operators/src/opensdl_operators/tools.py:34-54 publishes five tool names that appear nowhere else in the repository; MCP registers five different names. The declared input_schema values have required with no properties. An agent that trusts this endpoint builds calls that cannot be dispatched.

F3 — Declared timeouts do not bind a blocking adapter [verified] · M

asyncio.wait_for can only interrupt at an await point. The shipped local-compute adapter is async def with zero awaits — pure synchronous computation in a coroutine. Measured: declared timeout 0.1 s, elapsed 2.00 s, no TimeoutError.

Timeouts work for genuinely async adapters and are advisory for anything that blocks, which is what a real vendor SDK will do. The same property makes max_concurrency illusory. README.md:22 lists timeouts among what works.

F4 — CLI diagnostics are unusable for ordinary mistakes · M

Of roughly twelve CLI error paths exercised, exactly two produce a clean message — both in twin, the only group written with explicit try/except. Everything else dumps 24–164 lines of Rich traceback through framework internals, exposing absolute virtualenv paths. A workflow step naming an unknown capability produces WorkflowExecutionError: sim.mix_colour — the bare identifier, with no "unknown capability" and no list of available ones; over HTTP the same becomes 400 {"detail":"sim.mix_colour"}. Exit codes carry no information: denial, timeout, not-found and crash all exit 1. twin is a working in-repository template for the other fifteen commands.

F5 — Policy denial is HTTP 400 and the typed decision is discarded · S

app.py:122-123,144-145 are bare except Exception: raise HTTPException(400, str(exc)), so denial is indistinguishable from malformed input; it should be 403, resource-busy 409, timeout 504. PolicyDecision carries rule_id and policy_version and engine.py:243-245 flattens it to an f-string, so no caller can learn which rule denied them. Denial is persisted as failed — there is no denied state. OpenAPI documents only 200 on every route, so the 400/404/409 the code returns are invisible to generated clients.

F6 — Read-only inspection creates and writes the store · M

system.py:74-75,119-123 calls database.initialize() unconditionally and then upserts every capability and resource. Running opensdl inspect against a nonexistent run with no database present created .opensdl/opensdl.db with eight tables, eight capability rows and four resource rows. This affects inspect, events, export, capability list, twin project, doctor and run. Backlog §3.1 proposes a new safe command; it does not record that the seven existing ones are unsafe.

F7 — MCP is never installed or executed · S

The mcp extra is not passed by Makefile or CI, so mcp_available() is false in the project environment. build_mcp_server has no caller and there is no serve-mcp command; the only test patches the import to raise. The five tool registrations, including the write path execute_capability, have never run.

F8 — Two divergent Python surfaces · M

opensdl.OpenSDLClient (HTTP, 12 methods) and opensdl_controller.OpenSDLSystem (in-process, full runtime, the only way to reach campaigns) are both "the Python SDK" depending on which document you read. Also: workflow submission is asymmetric — the CLI takes a path, the API and SDK require the whole workflow inlined as JSON on every call, because there is no server-side workflow registry.

G. Documentation drift and dead contracts

G1 — Stale numbers shipped in `32a6c12` [verified] · S

README.md:79 — "spans 960 frames over 40 seconds" → 1176 frames, 49 s
VALIDATION.md:41,83 — "15 tests" overlay → 16
docs/development/backlog.md:131 — "a 40-second authored sequence, and 70 scene checks" → 49-second, 108 checks

propagation.yaml:124-126 declares examples/** → documentation precisely to catch this, and nothing invokes it (D2). The tool built to prevent this drift let it ship.

Resolved 2026-08-06, and the recurrence is the point. A further scene rebuild moved the counts again, and a manual sweep then found more stale references than this finding listed — docs/architecture/digital-twin.md, docs/architecture/lab-onboarding.md and a second figure in VALIDATION.md had all drifted as well. Volatile counts were removed from the backlog's completed ledger entirely, since a record of finished work should not need editing every time the artifact behind it is rebuilt. The underlying cause is untouched: D2 remains open, and until opensdl propagate is wired to something this will drift a fourth time.

G2 — Public models that are exported, schema'd, advertised, and never constructed · S

Observation, AuthorizationReceipt, and Incident (packages/core/src/opensdl_core/models.py:202,224,237) have no table, no writer, and no runtime path. README.md:19 advertises "observations, decisions, authorizations, and incidents". ExecutionRequest.authorization_id is never populated, so the dispatch record has no link to an authorization — one of the seven records SAFETY.md requires. CampaignDefinition is the same case (B7).

RunState.PAUSED, ABORTING, ABORTED, and TaskState.WAITING_FOR_AUTHORIZATION are never written by any code path; ABORTING is only read, at engine.py:362. CapabilityAdapter.abort() has zero callers and supports_cancellation has zero readers, while docs/concepts/capabilities.md:11 presents abort as implemented — so an adapter author wiring real hardware will implement it and it will never fire.

ReplayAdapter (packages/simulation/src/opensdl_simulation/replay.py) and build_run_graph (packages/provenance/src/opensdl_provenance/graph.py:29) have zero callers and 50%/0% coverage, while docs/index.md lists replay as a design goal and ROADMAP.md lists research-graph projection as implemented.

G3 — Claims to soften · S

SQLite and PostgreSQL-compatible — portable types, zero Postgres tests, no Postgres CI; the only dialect-specific code is SQLite-specific. VALIDATION.md:146 concedes this; README.md does not. safe local numerical-analysis — safe by absence of an evaluator, not by a guard. structured human-task attestations — echoes caller-supplied completed_by and stamps "attested": true on an unauthenticated string, which reads as a stronger claim than the data supports in any audit context. closed-loop … grid optimizer — observe() is an explicit no-op, so it is exhaustive enumeration. materials, chemistry, physics extension packs — 10 models, ~78 lines, no units, no capabilities, no committed schemas. unit, integration, end-to-end, and conformance tests — the marker taxonomy makes three of those four nearly empty (D5).

Cross-cutting patterns

The twin subsystem is the quality bar. It is the only command group with clean one-line CLI errors, differentiated HTTP status codes, content-addressed ETags, adversarial path-traversal tests, a digest-pinned contract that refuses a mismatch, and 95–100% coverage. Several findings above reduce to "make the rest of the codebase look like twin", which is a much cheaper framing than the raw count suggests: F4, F5, C7 and D4 all have a working in-repository template.

The schema was designed correctly and the wiring was never finished. EventRecord.campaign_id is an indexed column with an interface-level filter and repository implementation, and no caller. lifecycle.py is a complete, tested state machine with no callers. validate_reference_adapter_plugins is thoughtful supply-chain code with no callers on the mandatory path. abort(), supports_cancellation, binding.config, ExecutionResult.metadata, AuthorizationReceipt — all present, all unreachable. Many P0 items here are therefore S, not redesigns.

The documentation is honest where it was written deliberately and stale where it was written once. SAFETY.md, VALIDATION.md's non-claims, and the backlog are better than the software they describe. README.md and SECURITY.md's "Secure defaults" are the two places where the present tense outruns the code.

Coverage percentages are actively misleading here. Policy reports 100% and tests almost nothing (D3); the reproducibility test reports green and does not run (D1); conformance reports passed and constrains nothing (D4). Any prioritisation driven by coverage numbers would pick exactly the wrong targets.

Suggested first moves

Cheap, verified, and each closes a gap between what the documents promise and what the code does:

.env.* in the generated .gitignore (C1) — two lines.
Guard the resume path against re-dispatching a non-SUCCEEDED task (A1) — a few lines, and it is the one finding with a physical-safety character.
Move output-schema validation outside the retry loop (A2).
Thread environment and operator from the manifest into CampaignRunner (B2).
Fix the three stale numbers (G1) and wire opensdl propagate into make lint (D2) so the next ones cannot ship.
Install Blender in CI, or gate the skip behind an environment variable that converts it to a failure (D1).
Remove or hard-reject binding.config; docstring abort() as reserved; correct capabilities.md:11; relabel attested (C2, G2).
Thread campaign_id through run_workflow into _emit (B5) — before more runs accumulate without it.

Items 1–4 and 7–8 are all small and all currently mis-describe the system to its users.