Bind each accepted marker to the exact workflow run attempt, immutable artifact, source comment, and current PR head so a historical successful run cannot authorize a different payload.
Address P1 finding: extract_payload now strictly requires the parsed JSON
to be a dict, and validates each field's type and format:
- pr_number: string of digits
- pr_head_sha: 40-char lowercase hex string
- products: alphanumeric with commas/dots/hyphens/underscores only
- run_id: string of digits
- cases_ref: string (may be empty)
validate_run_id also guards against non-string input.
Added tests for: integer/array/string/null JSON, numeric field types,
invalid SHA format, injection in products, missing required fields.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Address P1 lint finding: structured eval-dispatch comments could be
forged by unauthorized users. Add three-layer consumer-side validation:
1. comment.user.login == 'github-actions[bot]' (platform-enforced identity)
2. comment.performed_via_github_app.slug == 'github-actions' (App signature)
3. payload.run_id verified against actual successful workflow run via API
Also adds:
- eval_poll_validate.py: consumer validation module (in-repo, auditable)
- test_eval_poll_validate.py: unit tests proving forged comments are rejected
- Go security contract test updated to assert run_id and validate reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- /eval on one's own PR may omit sha=: the guard auto-pins the
dispatch-time head (commenter == PR author leaves no third-party
swap window); dispatching another author's PR still requires the
explicit reviewed SHA (keeps the P1-2 TOCTOU remedy where the
threat lives)
- cases= is now validated structurally per git check-ref-format
semantics (leading/trailing//double slashes, '..', dot-leading
components, .lock suffixes) and rejects '-'-leading values to
prevent git fetch option injection (review P2)
Users listed in .github/eval-allowlist.txt (default branch, PR-reviewed)
may dispatch /eval for their own PRs only; write/maintain/admin retain
dispatch for any PR. Fail-closed on permission API 404/network errors.
Auto-CR (P1) correctly flagged that the carve-out's "nothing else changed"
guard was keyed on len(otherFailures) == 0, which only observes changes the
gate already judges incompatible. Several parameter contract changes are
individually compatible and so produce no failure at all: relaxing required or
cli_required, clearing required_when, widening enum, clearing interface_type,
and clearing property through a reviewed mapping exclusion. Any of those could
have ridden along with a reviewed type migration, leaving the exemption wider
than both its documentation and what the entry actually reviewed.
Replace the failure-list heuristic with a real equality check over every
published field except Type. Comparing the struct also means a field added to
parameterSchema later is covered automatically, instead of silently widening
every existing entry. The type check moves back ahead of the field loop because
it no longer needs to observe the other findings.
Add a rejection case for each individually-compatible direction. Each case first
asserts that the drift alone really is compatible, so it keeps exercising the
equality guard instead of quietly duplicating one of the incompatible-bundle
cases.
Verified against the previous implementation: with the old guard all six new
cases fail while the nine incompatible-bundle cases still pass, which is exactly
the gap that was reported.
Address the two P1 review findings and the coverage-gate CI failures:
- Every install/upgrade path that removes a skill dir (opposite-mode
leftovers, stale dingtalk-* / dws-shared, and same-name refreshes) now
moves the directory to ~/.dws/skill-backups/<stamp>/ first across
install.sh, install-skills.sh, install.ps1, install.js, `dws skill
setup`, and `dws upgrade`. A backup failure preserves the original
directory and never removes it.
- Remove --yes from every copyable `dws skill setup` example and document
what the command may remove; add regression tests that declining the
confirmation performs no removal and that the confirmation previews
every directory slated for backup+removal.
- Rename the skill-mode tests to the TestCrossPlatformCoverage* prefix so
the platform coverage gate selects them, and add edge tests for the
backup/prune/cleanup fallback branches, restoring changed-code coverage
to 100%.
Co-authored-by: Cursor <cursoragent@cursor.com>
schema-compatibility is the third check in the same Interface Integrity job,
after the two CLI interface gates. It also rejected every published parameter
type change outright. Because the earlier gates failed first and `set -e`
stopped the step from ever running, this one never surfaced in CI, so the
previous exemption only covered two thirds of the problem.
checkParameterCompatibility now consults a precise allowlist: the tool path,
parameter name and both type values must match exactly, making it
direction-sensitive by construction, and it applies only when nothing else the
gate checks about the parameter moved (default, interface_default, format,
property, interface_type, required, cli_required, required_when, enum). The type
check moved to the end of the function so the carve-out can see those findings;
ordering is unobservable because the result is sorted.
The only entry is "minutes/minutes.apply_minutes_permission" parameter "policy"
migrating from "string" to "integer" (for #912). That type is projected from the
Cobra flag type (provenance cobra_flag_type), so it describes how the CLI accepts
a value. Consumers build a command line from it, and "--policy 4" is the same
argv under either declaration — a quoted "--policy \"4\"" still reaches pflag as
4 — while RunE keeps enforcing the same [2,4] domain. The parameter maps to
property "policyId", which the command has always sent as a number, so "integer"
is closer to the actual request than "string" was.
Table values must be the canonical form schemaType emits: the JSON encoding of
the type keyword, so `"string"` with its quotes rather than a bare string. The
guard test recomputes both through schemaType and checks the decoded name
against the closed JSON Schema type set — reviewedInterfaceRefRedirect was
silently disabled twice by exactly this class of spelling mistake.
mergedFlagContractOtherwiseChanged was only ever exercised on the path where
every condition holds still, because `||` short-circuits: with no reviewed
entry the first operand already decides the outcome and the function is never
called at all. That left its five regression branches uncovered and put
changed-code coverage at 88.0952% against a 100% target.
Add the merge-path counterpart of the checkCompatibility bundled-regression
table, pairing each of shorthand / required / hidden / no-opt / scope with the
reviewed type change and requiring the type failure to reappear. Changed-code
coverage is now 100%.
The authoritative interface baseline and command-compatibility gates
rejected every flag type change on a historical command, with no review
channel — even when the new type only moves the same validation from RunE
to flag parsing. Both now consult a precise allowlist.
An entry must match command path, flag name and both type names exactly,
so it is direction-sensitive by construction, and it applies only when
nothing else about the flag moved (shorthand, required, hidden, no-opt,
scope). A bundled regression re-reports the type change.
The first and only entry is "dws minutes permission apply --policy" moving
from string to int (for #912): the old RunE parsed with
strconv.ParseInt(v, 10, 64) and enforced [2,4], the new one lets pflag
parse with base 0 and still enforces [2,4], so the historical set of
successful invocations is a subset of the new one. Base 0 additionally
accepts spellings like "0x3", which widens rather than narrows. Defaults
are excluded from the guard because the migration necessarily changes one.
In the snapshot gate the exemption resolves against the canonical
Command.Path, never the alias-expanded accepted path: an aliased command is
compared once per accepted spelling, so keying on that would let every
alias bypass the table.
The table is duplicated because check-authoritative-interface-baselines.sh
copies the whole scripts/policy/interface-baseline directory into a
worktree checked out at a historical revision and builds it there, so that
copy cannot import a package this branch adds. A guard test fails if the
two copies drift.
Upstream reorganized the multi-skill layout (#887: long-tail skills folded
into dingtalk-misc, dws-shared renamed to dingtalk-shared). Conflict
resolution keeps this branch's multi-by-default semantics (install.sh /
install.ps1 / skill setup default to multi; interactive prompts list multi
first) and adapts the cleanup paths to the rename: cleanup predicates now
recognize both dingtalk-shared (new bundle name, covered by the dingtalk-
prefix) and the legacy dws-shared so full installs and mode switches remove
pre-rename leftovers.
Co-authored-by: Cursor <cursoragent@cursor.com>
The allowlist added in the previous commit keyed the reviewed
sheet.range_set_style migration by bare RPC name ("update_range"), but
interface_ref holds the canonicalized JSON that parseTool produces via
canonicalRawJSON. The lookup therefore never matched, the carve-out was
effectively disabled, and the real gate failed with
`schema tool "sheet/sheet.range_set_style" changed interface_ref`.
The existing redirect test did not catch this: it registered the fixture entry
using its own value and then asserted with the same value, so any format would
have passed. That is the same mistake as keying a probe on the author's field
spelling instead of the wire contract.
- The allowlist entry now uses the compact canonical JSON, taken from the gate's
own output rather than from pretty-printed `dws schema`.
- TestCrossPlatformCoverageReviewedRedirectKeysAreCanonicalJSON recomputes every
registered key through canonicalRawJSON, so a bare name or a pretty-printed
variant fails locally instead of only in CI.
sheet export keeps its reviewed mcp + submit_export_job declaration. The comment
now records the known trade-off explicitly: --export-format csv is a mutually
exclusive branch that reads get_range_as_csv and never invokes the declared
submit_export_job, so this declaration does not cover the csv branch's backing
interface. Attributing that branch is left out of scope for this change.
Verified against the real gate, not just unit tests: all four Interface
Integrity checks pass (authoritative-interface-integrity,
check-command-compatibility, schema-compatibility, skill-command-integrity).
Two review findings, plus a same-class defect found by sweeping for it.
1. compatibleInterfaceRefRedirect accepted any mcp tool repointing from any
non-empty interface_ref to any other, as long as no other check for that tool
failed. Schema shape cannot prove two RPCs share business semantics,
permissions, error behaviour, or side effects, so that would have let every
future backend swap bypass the gate it exists to enforce. It is now keyed on
an explicit reviewedInterfaceRefRedirect allowlist of exact tool + old→new
pairs, currently holding only the reviewed
sheet.range_set_style: update_range → set_cell_range migration. Every other
ref change is reported again.
2. sheet export --export-format csv only printed a stderr warning when
get_range_as_csv returned hasMore=true, then wrote --output and reported
success with exit code 0. Automated callers, and anyone not watching stderr,
would treat an incomplete file as a complete export, and an existing target
file was overwritten with truncated data. Truncation now fails before the
write (leaving any existing file untouched) unless --allow-truncated is
passed; with the opt-in the success line states the data is incomplete.
--allow-truncated is registered in the reviewed mapping ledger as a local
policy input.
3. Swept for the same classes and found sheet_dimension.go repeating the
fmt.Sscanf("%d") prefix-parse hole in three places: insert_dimension,
delete_dimension, and update_dimension all accepted --length "3x" as 3, so a
malformed value silently operated on the wrong row/column count — the delete
direction is not rollbackable. All three now use strconv.Atoi. (Checked and
cleared: sheet csv-get also surfaces hasMore, but it has no --output and only
returns the flag in its JSON payload, so it is not the same fail-open shape.)
Tests: allowlist rejection cases (unreviewed target ref, and the same pair on a
tool absent from the allowlist, asserted outside the table so the registration
survives until checkCompatibility runs); truncation fail-closed with a
pre-existing output file asserted byte-for-byte unchanged; --allow-truncated
write-through; and an untruncated read needing no opt-in. Changed-code coverage
stays at 100% (939 statements).
Dennis