Bind coverage repair to the stable PR head and merge identity plus protected-main containment without treating the live base SHA projection as permanent identity.
Add semantic regression coverage for base advancement and update the governance documentation.
GitHub omits merge-related repository settings from tokens without Contents write. Accept only the exact dual omission in read-only admission, and require the dedicated App to observe the reviewed values before any auto-merge mutation.
The smoke test creates temp home directories with .config/kimchi markers
for agent detection. On Linux CI runners XDG_CONFIG_HOME may point to the
runner's real config path, causing resolvedAgentTargets to look outside
the temp home. Unset it so detection resolves against the test's temp dir.
Restrict backup pruning to directories whose names match the DWS stamp
format (YYYYmmdd-HHMMSS with optional -N suffix) across all 8 installer
surfaces (Go, npm, 4 shell, 2 PowerShell). Unknown directories in
~/.dws/skill-backups are now preserved. Also fixes Windows coverage test
portability and covers the remaining macOS changed-code gap (retire
warning loop in runUpgrade).
The five internal/app partitions ran end to end inside one job, so the app
shard's wall clock was the sum of all five: 780s in CI, of which the schema
partition owned 357s. Each partition is now its own matrix shard, so they run
concurrently and the shard's wall clock is set by its slowest partition rather
than by their total. Every partition shard still selects the same single
internal/app package, so the impacted-package query maps the shard name back to
app and the partition only chooses which tests run.
The helper gains a partition argument and a list-partitions mode. APP_PARTITIONS
is the single source of truth for the set, and the discovery pass still runs in
every job, so each one independently verifies that the partition patterns cover
every top-level test exactly once before running the one it was asked for.
Two fail-closed checks guard the split, because the helper's own coverage check
can no longer prove the whole package ran once the partitions are separate jobs:
- The helper cross-checks APP_PARTITIONS against the coverage counters in both
directions, so a counted partition that nothing dispatches and a dispatchable
partition with no counter both fail instead of silently skipping tests.
- TestCIAppRacePartitionMatrixMatchesHelper pins the workflow's app-<partition>
shards to list-partitions output in both directions, so a partition cannot
lose its job while every job stays green.
The discovery loop variable is renamed from partition to spec: it would
otherwise shadow the partition requested on the command line, which run mode
reads after the discovery pass completes.
The workflow contract pinned the focused path by literal: the job name
`Test (changed packages)`, the unsharded
`list "$TEST_BASE_REF" "$TEST_HEAD_REF"` call, and a single
`go test -timeout=15m` line standing in for internal/app's package-level
headroom. Sharding the job changed all three literals, so `Test (workflow
and release contracts)` failed on this branch even though every shard
selection test passed.
Each invariant the contract guarded still holds, so the assertions are
updated to the new shape rather than relaxed:
- the focused job must still exist, now as the matrix job, named the way
the contract already names `Test (race: ${{ matrix.shard }})`;
- package selection must still derive from the authoritative synthetic
merge base/head, now with an explicit shard argument, so pointing it at
any other ref still fails the contract;
- internal/app's headroom is asserted through the process-isolating
helper and the per-shard budgets, mirroring the assertions already
applied to test-race. That is stronger than the old single -timeout: it
pins the mechanism that keeps the suite inside its budget rather than
the number alone. release-scripts membership is asserted too, because
its dedicated job only runs at full-suite or release-sensitive scope,
so losing it here would silently stop testing test/scripts changes.
The shard comparisons in the focused job are quoted so that job reads
verbatim like test-race's.
Ablating the implementation one change at a time turns the contract red
in all five cases: removing the app helper call, dropping release-scripts
from the matrix, selecting from HEAD~1, collapsing the matrix back to a
single unsharded job, and dropping the cli/smoke timeout budget.
Reading the package list with `mapfile < file` has unambiguous line
semantics. Routing it through a step output and a here-string instead
would append an extra empty array element if the value ever carried a
trailing newline, and that element would reach go test as an empty
package argument. The step output now carries only a single-line boolean,
and the list travels through RUNNER_TEMP. An explicit empty-entry guard
fails closed if the file is ever malformed.
This job cannot execute on its own pull request — editing a workflow
routes the revision to full_suite, which skips the focused path — so the
implementation deliberately avoids depending on platform-specific
trailing-newline behavior that local verification cannot observe.
The focused path tested every impacted package in a single job with a
plain `go test -race`, so internal/app ran inside one long-lived process
alongside all of its reverse dependencies. That is exactly the shape
scripts/ci/run-app-race-tests.sh exists to avoid: a single app test
process retains every constructed command tree in framework registries,
so the run grows to 900s and the job stays alive long enough to be
reclaimed by the runner. Recent focused runs failed with SIGTERM after
9-10 minutes without a single test failure, and one earlier run failed
at `internal/app 902.651s`, 2.65s past the package timeout.
Fan the same package plan across the shard matrix test-race already
uses, and run each shard the way test-race runs it: internal/app through
the process-isolating helper, cli/smoke with their wider package budget,
release-scripts without race and with archive tooling.
changed-test-packages.sh gains `list-shard`, which intersects the
impacted set with scripts/ci/test-packages.sh shard membership so shard
definitions stay single-sourced — and so an unknown shard name aborts
there rather than reporting an empty selection, which would let a
mistyped shard skip every test while reporting success.
release-scripts is in the matrix on purpose: its dedicated job only runs
at full-suite or release-sensitive scope, so omitting it here would stop
testing test/scripts changes altogether. A test pins that the shard
selections partition the impacted set exactly, so shard-plan drift
cannot silently shrink focused coverage.
The Coverage context was the PR critical path (~17 min end to end):
coverage-current re-ran the whole suite serially (-p 1, ~13 min) and
coverage-baseline re-ran it again at the merge-base (~13 min) although
that profile is a pure function of the base commit.
- coverage-current now owns only the scoped (standard-tier) profile;
full-suite candidate profiles come from a 5-way shard matrix
(app/cli/generators/helpers/remaining) that keeps -p 1 inside each
shard on isolated runners. scripts/ci/test-packages.sh list-coverage
defines the shards and verify proves the union equals the previous
single-run package set exactly once.
- the aggregate Coverage job reassembles the disjoint shard profiles
into coverage.txt before make coverage-gate, failing closed when a
shard file is missing, so gate semantics (100% changed-code +
scope-matched overall non-regression) are byte-compatible.
- coverage-baseline restores the merge-base full-suite profile from an
exact-key cache (merge-base SHA + resolved Go version) written by the
last green main push; any miss falls back to recomputing in the
merge-base worktree. Exact key only - no prefix fallback, a near-miss
profile would compare the candidate against the wrong commit.
- new contract tests pin the shard matrix, the assembly step, the
exact-key cache pair, and the absence of restore-keys; the package
plan test also covers the coverage shard partition.
Bind each accepted marker to the exact workflow run attempt, immutable artifact, source comment, and current PR head so a historical successful run cannot authorize a different payload.
Address P1 lint finding: structured eval-dispatch comments could be
forged by unauthorized users. Add three-layer consumer-side validation:
1. comment.user.login == 'github-actions[bot]' (platform-enforced identity)
2. comment.performed_via_github_app.slug == 'github-actions' (App signature)
3. payload.run_id verified against actual successful workflow run via API
Also adds:
- eval_poll_validate.py: consumer validation module (in-repo, auditable)
- test_eval_poll_validate.py: unit tests proving forged comments are rejected
- Go security contract test updated to assert run_id and validate reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The GitHub Actions runner cannot reach internal Aone CI API (structural
network isolation). Replace the curl-to-internal step with a structured
HTML comment (<!-- eval-dispatch: {...} -->) that an internal Devix
polling service picks up every 3 minutes to trigger the Aone CI pipeline.
This eliminates the EVAL_TRIGGER_URL/EVAL_TRIGGER_TOKEN secrets dependency
from the GitHub side — those can be removed once verified.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Users listed in .github/eval-allowlist.txt (default branch, PR-reviewed)
may dispatch /eval for their own PRs only; write/maintain/admin retain
dispatch for any PR. Fail-closed on permission API 404/network errors.
chichuan