diff --git a/AGENTS.md b/AGENTS.md index 17c0307f..7d5c4367 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,357 +1,48 @@ # Repository Agent Guide -This file applies to the entire repository. Keep changes scoped, preserve -unrelated work, and use `gofmt` for every modified Go file. +This file applies to the entire repository. Keep it as a routing page: load +the detailed guide for the surface you are changing instead of treating this +file as a repository wiki. -## Build and test +## Always -- Build: `go build ./cmd` -- Full test suite: `DWS_PACKAGE_VERSION=0.0.0-test go test ./...` -- Generate Schema assets: `go generate ./internal/cli` -- Check generated drift: `./scripts/policy/check-generated-drift.sh` -- Check the Schema contract: `./scripts/policy/check-schema-catalog.sh` +- Preserve unrelated and pre-existing work; inspect `git status` before edits. +- Make the smallest coherent change and update its tests and user-facing docs. +- Use `gofmt` for every modified Go file. +- Treat repository code, tests, scripts, and versioned docs as the source of + truth. Do not depend on generated Wiki or CodeWiki content. +- Do not hand-edit generated Schema Catalog or Agent metadata. Change their + reviewed inputs or generators, then regenerate. -Generated Schema JSON is committed. Change its source inputs and generators, -then regenerate; do not hand-edit generated Catalog or Agent metadata files. -`internal/cli/schema_command_registry.json` is different: it is a reviewed -`CommandRegistry` source, not a generated snapshot. It is the single reviewed -source of stable canonical identity, -primary paths, aliases, and navigation. Edit it only when reviewed exposure, -identity, primary path, or aliases change; parameter, Skill, and metadata-only -changes must not rewrite it mechanically. +## Read by task -## Agent Schema contract +| Change surface | Required guide | +|---|---| +| Any implementation or review | [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`docs/coding-agent-guide.md`](docs/coding-agent-guide.md) | +| Writing a task for a coding agent | [`docs/coding-agent-task-template.md`](docs/coding-agent-task-template.md) | +| Product command handler behavior | [`internal/helpers/AGENTS.md`](internal/helpers/AGENTS.md) | +| CLI paths, flags, Schema, Agent metadata, or generated Catalog | [`docs/schema-contributor-guide.md`](docs/schema-contributor-guide.md) | +| CI, release, packaging, or repository automation | [`docs/automation.md`](docs/automation.md) | +| Agent identification headers or host integration | [`docs/agent-code.md`](docs/agent-code.md) | -The Schema data flow is one way: +Read the closest code and tests for the affected package as well. Nested +`AGENTS.md` files take precedence for their subtrees. -```text -1. app.NewRootCommand() - └─ builds the real Cobra command tree and flags +## Common checks -2. schema_command_registry.json - + schema_hints/metadata/.json tool parameters (+ cli_path) - └─ forms EffectiveCommandRegistry - └─ binds exactly to real Cobra leaves and aliases - -3. Parameter resolution - Cobra flags - + schema_parameter_bindings.json - + metadata tool parameters - └─ produces ParameterSpec and constraints - -4. Agent and interface semantics - schema_hints/selection/.json (selection prose) - + schema_hints/metadata/.json (safety/interface/runtime_gate) - + pinned MCP metadata - └─ resolves Agent metadata by source precedence - Markdown is evidence only; it is not concatenated into final prose - -5. One typed hub - BoundCommandRegistry - + ParameterSpec - + Agent metadata - + Interface metadata - └─ resolves every command exactly once into ToolSpec - └─ aggregates SchemaRegistry + SchemaIndex - -6. One-way publication - SchemaRegistry - └─ internal/cli/schema_catalog.json - └─ dws schema list/product/group/leaf/--all -``` - -Parameter overlays from metadata are merged into `EffectiveCommandRegistry` -*before* Cobra binding; after that point there is no second identity source and -no identity precedence winner. The binder must reject a missing/non-runnable -Cobra path, an alias collision, and any native identity annotation that -disagrees with the effective registry. A missing native identity annotation is -allowed because annotations are implementation-side assertions, not identity -fallbacks. - -The assembler resolves every bound command exactly once into one `ToolSpec`. -Build-time gates and the snapshot serializer consume that source-resolved typed -registry/index. Runtime projections and delivery gates consume the typed -registry/index returned by the production snapshot loader. Neither path may -reopen annotations, merge source records, or use a previous Catalog or other -generated JSON as a source. `schema_catalog.json` is output-only in the -generation graph. The production loader decoding the embedded published -snapshot is a delivery boundary, not source resolution; it must never create or -repair a Cobra command, flag, registry entry, or later Catalog generation. - -This split is architecturally isomorphic to Lark's typed metadata registry, -navigation catalog, and schema renderer. DWS intentionally preserves its -existing flat JSON wire contract for compatibility; do not treat architectural -alignment as permission to make an unversioned wire-format change. - -The reviewed `CommandRegistry` is the sole source of stable command identity -and navigation. The executable Cobra tree remains the source of truth for -whether a CLI path exists, is runnable, and which flags it accepts. Schema -coverage is bidirectional: - -1. Every final `SchemaRegistry` tool, including its serialized Catalog - projection, must resolve to an executable Cobra command. -2. Every public runnable Cobra leaf must either resolve to Schema or appear as - an exact, reviewed exclusion with a non-empty reason in - `internal/cli/schema_command_exclusions.json`. - -Do not use prefix or wildcard exclusions: they can silently hide future -commands. Remove an exclusion when its command enters Schema; stale, invalid, -or duplicate exclusions must fail generation and CI. - -When adding or changing an Agent-visible command, review all relevant inputs: - -- `internal/cli/schema_command_registry.json` for the reviewed - `CommandRegistry`: canonical identity, primary CLI path, aliases, and stable - navigation. It is the identity source and is not a generated artifact. -- `internal/cli/schema_command_registry.schema.json` is its closed, - machine-readable editing contract. Preserve the local `$schema` reference; - unknown fields, invalid visibility values, stale paths, and collisions fail - Go validation and policy. -- `internal/cli/schema_hints/metadata/.json` for safety, interface, - `runtime_gate`, and optional parameter overlays (`parameters` / `cli_path`). -- `internal/cli/schema_hints/selection/.json` for reviewed Agent - selection prose (`agent_summary`, `use_when`, `avoid_when`, `examples`). -- `internal/cli/schema_hints/index.json` only maps product IDs to those files. -- Native Runtime Schema identity annotations, when present, as consistency - assertions against `EffectiveCommandRegistry`. They must agree exactly and - must never materialize, infer, or override registry identity. -- Flag-to-interface property mappings and required/default semantics. -- Generated files under `internal/cli/schema_agent_metadata/` and - `internal/cli/schema_catalog.json` after running generation. - -Run the reverse-completeness tests whenever the Cobra tree changes. A command -that works through `dws ` but cannot be found through the matching -`dws schema` lookup is a contract failure unless it has a reviewed exact -exclusion. - -Metadata parameter overlays must reference an exact public runnable Cobra leaf -and real flags. They may override Schema description, interface-property/type -mapping, `required`, and `required_when`; they must not create commands or -flags, define an interface, or advertise an unknown RPC. Every authored entry -requires `reviewed: true` and a non-empty review reason. - -For Agent-authored metadata or selection edits: - -1. Confirm the exact command and flag names in the current Cobra tree. -2. Edit only the owning block (`metadata/` or `selection/`); do not mix fields. -3. Add the smallest possible entry; do not copy generated Catalog fields into - the input. -4. Describe user-visible semantics in `review_reason` and parameter - descriptions. -5. Run generation, drift, Schema policy, and the focused CLI tests before - proposing the change. - -## Agent curation workflow (Schema hints) - -Use this workflow when refreshing Agent selection prose and confirmation -alignment. Prefer **agent-authored review** over bulk merge scripts that dump -`selection-review.json` or Skill Markdown into Catalog fields. - -Human-authored inputs are split into two blocks: - -| Block | Path | Owns | -|---|---|---| -| **metadata** | `internal/cli/schema_hints/metadata/.json` | `effect` / `risk` / `confirmation` / `idempotency` / `interface_*` / `runtime_gate` / optional `parameters` | -| **selection** | `internal/cli/schema_hints/selection/.json` | `agent_summary` / `use_when` / `avoid_when` / `examples` (+ product routing) | - -`index.json` only maps product IDs to those files. Do not mix selection fields -into metadata files or metadata fields into selection files. - -### Goals - -1. **Selection prose** is decision-oriented (Feishu/Lark style): trigger intent, - sibling-command routing, and outcome shape — not a restatement of the - summary. Delivered Catalog provenance is `reviewed_explicit` from - `selection/`. -2. **Safety** follows Runtime: `confirmation=user_required` iff the tool's - metadata `runtime_gate != none` (for example `confirm_delete`, `typed_yes`, - `confirm_dangerous`). -3. **Parameter overrides** (former Manual `commands`) live on metadata tools as - `parameters` (+ `cli_path`) and are applied into EffectiveCommandRegistry. - -### Authoring - -For every curated tool: - -1. Edit `metadata/.json` for safety/interface/gates/parameters. -2. Edit `selection/.json` for selection prose (`reviewed: true`, - `review_reason`, `source_refs`). -3. Run `make generate-schema`. Do not hand-edit generated - `schema_agent_metadata/` or `schema_catalog.json`. - -### Pull live MCP descriptions (personal token) - -Pinned `internal/cli/schema_mcp_metadata.json` is a sanitized baseline. Prefer -live Schema from a logged-in personal session: +Choose checks from the matrix in `docs/coding-agent-guide.md`; do not claim a +check that was not run. ```bash -dws auth status # token_valid should be true -dws cache refresh # refresh discovery / tools cache -dws schema -f json -# or CLI path: dws schema --cli-path "drive copy" -f json +make coding-agent-harness +make build +make format-check +make test +make policy +git diff --check ``` -Resolve MCP identity via `interface_ref` when CLI canonical ≠ MCP path -(example: CLI `drive.copy_document` → live `doc.copy_document`). On pull -failure, fall back to Skill + Cobra Help + pinned MCP, and record evidence -(for example `live-dws-schema:#FAILED`). Never print or commit tokens. - -Precedence when sources disagree: **Runtime/Cobra > live MCP > pinned MCP > -Skill (evidence only)**. - -### Parallel product agents - -Split work by product groups. Each agent must: - -- Read Skill, Cobra/`--help`, Runtime confirmation sites, and live `dws schema` - for its tools. -- Hand-write selection + metadata; forbid wholesale JSON merges from review - dumps. -- Edit only its `metadata/.json` and `selection/.json`. -- **Never** `git checkout` unrelated product files to “clean scope”. - -### Regenerate and gates - -```bash -make generate-schema -./scripts/policy/check-runtime-confirmation-truth.sh -go test ./internal/app -run '^TestSheetFinalSchemaConfirmationMatchesRuntimeGuards$' -count=1 -``` - -Example rules (fail generation otherwise): - -- At most two examples per tool; no `--yes` in stored examples. -- Examples must match live Cobra argv (path, flags, required groups). -- No shell comments in examples. - -After generation, spot-check Catalog: selection provenance is -`reviewed_explicit` from `selection/`, and `user_required` count equals -metadata `runtime_gate != none`. - -`make generate-schema` is a full deterministic snapshot rebuild, not an -incremental patch over the previous Catalog. It rereads every reviewed input, -removes stale generated product metadata, and rewrites the exact metadata and -Catalog projections. Incremental work happens only when an Agent or human -edits selected `metadata/` or `selection/` entries; the next publication still -recomputes all outputs. Generated files must never be read back as merge input, -and byte guards fail generation if it changes the hint inputs or CommandRegistry. - -Selection prose may choose a more or less restrictive recommendation. It cannot -create a Cobra command or flag, change parameter facts, invent an -RPC/interface, alter safety metadata, or bypass command completeness. Examples -must use an executable primary/alias path and flags accepted by the live Cobra -command; never add `--yes` to stored examples. - -Every example is always checked against its real `BoundCommand`: exact path, -accepted flags, Cobra required flags/positionals, and the effective -`require_one_of`, `require_together`, and `mutually_exclusive` constraints must -all pass before execution eligibility is considered. A missing required value, -constraint failure, runtime error, or MCP resolution error is a contract bug; -none is a valid reason to skip an example. - -Example execution defaults to contract validation only. Runtime execution is -opt-in: an example enters `dry_run` only when its final `ToolSpec` publishes an -explicit reviewed dry-run capability. The test never injects `--yes`, and -`risk`/`confirmation` values do not manufacture preview support. A narrow -runtime precondition that cannot be derived from the typed contract may use an -exact zero-based `example_dispositions` entry with `mode=contract_only`, -`reviewed=true`, one of the schema-enumerated reason codes, and a concrete -non-empty reason. Such a disposition may only narrow an explicit dry-run -capability; it cannot turn an ordinary contract-only example into a skip. -Duplicate, missing, and out-of-range indexes fail validation. Never catch a -dry-run failure and dynamically downgrade it to `contract_only`. - -Normal Go tests run the exhaustive contract gate. Run -`make test-schema-agent-examples` to additionally execute the eligible subset -through the real Cobra `--dry-run` path with isolated HOME and blocked proxies. -The test reports stable `total`, `contract`, `dry_run`, `contract_only`, -`reviewed_manual`, and per-reason counts; changing those counts requires a -review of the corresponding typed dry-run capability or manual disposition. -This target is also part of `make policy`. - -Treat every tool `use_when` entry as a reviewed positive selection scenario -whose expected result is that tool's canonical path, and every `avoid_when` -entry as a reviewed negative scenario that must not choose that tool. The -deterministic gate derives a typed evaluation fixture from these same fields; -it requires exact tool coverage, a real runnable `BoundCommandRegistry` -primary command, at least one positive and negative assertion per tool, and no -literal contradictory expectations. It does not claim that string matching -proves natural-language understanding. - -Semantic selection is an explicit opt-in live-model check. Run the smoke set -(one positive and one negative scenario per product) with -`DWS_AGENT_SELECTION_LIVE=1 ARK_API_KEY=... ARK_BASE_URL=... ARK_MODEL=... go test ./internal/app -run TestManualAgentSelectionArkLive -count=1`. -Add `DWS_AGENT_SELECTION_FULL=1` to evaluate every committed tool scenario, or -set `DWS_AGENT_SELECTION_CASES` to comma-separated fixture case IDs. Normal CI -never calls a model; its blockers remain the reproducible fixture, binding, -example, provenance, and final-delivery facts. - -The live evaluator sends only case IDs/scenarios plus one same-product -candidate table; expected/forbidden assertions stay local and must never be -included in the model prompt. Built-in Ark HTTPS bases are allowlisted. A -different HTTPS provider requires its exact base in -`DWS_AGENT_SELECTION_ALLOWED_BASE_URLS`; plaintext HTTP is accepted only for a -loopback test server so API credentials are never sent to an arbitrary clear -text endpoint. - -## Safety metadata - -Parameter and safety resolution is mostly source-precedence based and -value-neutral: do not choose a winner because one value looks stricter. A -higher-priority reviewed metadata/explicit source may intentionally raise or -lower description, mapping, `effect`, `risk`, `confirmation`, or `idempotency`. -Preserve all candidates and the selected source in provenance, and fail -same-precedence conflicts rather than silently merging them. - -`required` is the exception. Cobra `MarkFlagRequired` is a hard floor: the -final Agent projection must keep `required=true` and cannot be lowered by -manual/hint overlays. Overlays may still raise an optional flag to required. -`cli_required` continues to mirror the executable Cobra marker. - -For command text, reviewed `ToolSchemaHint` wins first, then command-specific -Cobra Help, then MCP metadata. Generic RPC prose may remain an unselected -provenance candidate (and parameter-level `interface_description`); it must not -overwrite a specialized leaf's title or description. - -For every delivered `ToolSpec` and `ParameterSpec` field, the provenance -winner value must exactly equal the delivered value. Checking only source, -count, presence, or hash is not a sufficient final-delivery invariant. - -The same resolved `ToolSpec` must drive every projection. The full leaf payload -must equal the corresponding tool in `schema --all` and the full Catalog tool. -Overview/product/group summaries and Catalog summaries must equal -`ToolSpec.ToSummaryPayload()`. An alias lookup may change only the view fields -`cli_path` and `is_alias`; it must not re-resolve or mutate the command -contract. - -This build-time rule is distinct from runtime drift handling. If shipped Help -and leaf Schema disagree, pass only flags accepted by Cobra. For conflicting -safety information, do not silently take the less restrictive behavior: use -the safer interpretation or stop and report the contract drift. - -Do not infer one safety field from another. In particular, `effect=destructive` -or `risk=high` does not mechanically rewrite `confirmation`; the final -precedence winner for each field is authoritative. When -`confirmation=user_required`, obtain confirmation before adding `--yes`. -Keep CLI confirmation behavior and Schema metadata consistent, and add a -semantic regression test through the final embedded loader/query delivery -path; a generator unit test or JSON count alone is insufficient. - -## Current Schema boundaries - -- `schema list` remains a progressive overview. `schema --all` is the stable - full-export contract: every final `SchemaIndex` tool must contain its - complete leaf parameters, constraints, and safety semantics, including an empty - `parameters` object for commands without flags. Keep it suitable for the #602 - compatibility baseline and fail rather than silently emitting a partial - export. -- `schema --all` is not normal command discovery. Use overview -> product/group - -> leaf for routine Agent work. `--compact` is supported for context-saving - projections, but a compact full export is not a complete compatibility - baseline. -- `dws --help` defines whether Cobra exposes a path and which flags the - executable accepts. A leaf Schema defines Agent selection, parameter mapping - and constraints, and safety/confirmation semantics. A conflict is contract - drift, not permission to guess. -- Schema and Help describe commands; neither returns DingTalk business data. - After discovery, execute the real read/search/list command to obtain data. +For Schema work, the minimum generation entry point is `make generate-schema`. +For CLI path or flag changes, also run +`./scripts/policy/check-command-surface.sh --strict`. Report failures, +environment limits, and unrun checks explicitly in the handoff. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 38b9e684..f67200aa 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,9 +10,13 @@ under the project [Apache License 2.0](./LICENSE). ## Before You Start 1. Read `README.md`. -2. Read the relevant docs under `docs/`. -3. Inspect the code and tests for the area you will change. -4. Decide the smallest safe change that satisfies the request. +2. Normalize the task and select checks with + [`docs/coding-agent-guide.md`](./docs/coding-agent-guide.md). +3. Read the relevant docs under `docs/`. CLI/Schema/Agent metadata work must + also follow + [`docs/schema-contributor-guide.md`](./docs/schema-contributor-guide.md). +4. Inspect the code and tests for the area you will change. +5. Decide the smallest safe change that satisfies the request. Maintainers and automation authors should also read `docs/automation.md` for repo-local release and agent workflow diff --git a/Makefile b/Makefile index b5131e7b..5b7b6029 100644 --- a/Makefile +++ b/Makefile @@ -8,7 +8,7 @@ POLICY_GOTMPDIR ?= $(DWS_POLICY_TMPDIR)/go POLICY_ENV = DWS_POLICY_TMPDIR="$(DWS_POLICY_TMPDIR)" GOTMPDIR="$(POLICY_GOTMPDIR)" GO_SOURCE_LIST = git ls-files -z --cached --others --exclude-standard -- '*.go' -.PHONY: all help build rebuild test test-plan lint format-check fmt policy edition-test interface-integrity authoritative-interface-integrity coverage-gate coverage-gate-platform update-interface-baseline reset-interface-baseline schema-compatibility skill-command-integrity cli-smoke mock-mcp-smoke test-schema-agent-examples generate-schema generate-schema-agent-metadata generate-schema-catalog package release release-pre release-stable changelog-pre changelog-stable publish-homebrew-formula setup-hooks +.PHONY: all help build rebuild test test-plan lint format-check fmt policy coding-agent-harness coding-agent-task edition-test interface-integrity authoritative-interface-integrity coverage-gate coverage-gate-platform update-interface-baseline reset-interface-baseline schema-compatibility skill-command-integrity cli-smoke mock-mcp-smoke test-schema-agent-examples generate-schema generate-schema-agent-metadata generate-schema-catalog package release release-pre release-stable changelog-pre changelog-stable publish-homebrew-formula setup-hooks all: setup-hooks fmt lint build test rebuild @@ -21,6 +21,8 @@ help: @printf " make format-check - Check all repository Go source files with gofmt\n" @printf " make fmt - Format all repository Go source files\n" @printf " make policy - Check the built dws plus open-source and Schema policies\n" + @printf " make coding-agent-harness - Validate coding-agent task intake, routing, and self-check contracts\n" + @printf " make coding-agent-task TASK= - Validate a filled coding-agent task contract\n" @printf " make interface-integrity - Check historical commands and help contracts still work\n" @printf " make authoritative-interface-integrity BASE_REF= - Check the Git-owned PR merge-base\n" @printf " make coverage-gate BASE_REF= - Enforce overall non-regression and 100%% changed-code coverage\n" @@ -86,6 +88,13 @@ policy: @$(POLICY_ENV) ./scripts/policy/check-schema-binary.sh @$(POLICY_ENV) $(MAKE) test-schema-agent-examples +coding-agent-harness: + @./scripts/policy/check-coding-agent-harness.sh + +coding-agent-task: + @test -n "$(TASK)" || { printf '%s\n' 'TASK is required, e.g. make coding-agent-task TASK=task.md' >&2; exit 2; } + @./scripts/policy/check-coding-agent-harness.sh -task "$(TASK)" + edition-test: $(GO) test -v -count=1 ./pkg/editiontest/... diff --git a/docs/coding-agent-guide.md b/docs/coding-agent-guide.md new file mode 100644 index 00000000..f957d2e3 --- /dev/null +++ b/docs/coding-agent-guide.md @@ -0,0 +1,107 @@ +# Coding Agent Workflow + +This is the task intake and self-check contract for coding agents working in +this repository. It is intentionally independent of external Wiki systems: +the checked-out repository is the execution context and evidence source. + +## 1. Normalize the task input + +Start from the copyable +[`coding-agent-task-template.md`](coding-agent-task-template.md). Keep one +primary outcome per task. Fill unknown fields from the issue, nearby code, +tests, and versioned docs; state any assumption that can affect behavior. + +For a saved, filled template, run `make coding-agent-task TASK=path/to/task.md`. +The local checker rejects missing required fields, unsupported task kinds, and +unresolved placeholders before implementation begins. + +Do not invent acceptance criteria that expand the requested behavior. Stop for +user input only when the unresolved choice would change externally visible +behavior, compatibility, destructive scope, credentials, or external state. + +## 2. Establish the baseline + +1. Run `git status --short` and identify pre-existing changes. +2. Read the applicable guides linked from the root `AGENTS.md`. +3. Locate the implementation and its closest tests with `rg`/`rg --files`. +4. Reproduce the bug or capture the current contract before changing it. +5. Pick the smallest owning layer; avoid duplicating policy in a caller when a + shared typed layer already owns it. + +Never clean, overwrite, stage, or reformat unrelated user changes. If a +required file is already modified, inspect the overlap and preserve both +intents or stop with the exact conflict. + +## 3. Implement from authoritative inputs + +- Go behavior belongs in the package that owns the contract, with focused + tests beside it. +- Public CLI paths and flags must match the live Cobra tree and compatibility + policies. +- Schema and Agent-facing changes start from reviewed source inputs; generated + outputs are publication artifacts. +- Documentation describes behavior that exists in the same change. +- Secrets, tokens, local identities, and private endpoints must not enter code, + fixtures, logs, or handoff output. + +For generated files, run the repository generator and inspect the resulting +diff. A large or unrelated generated diff is a signal to stop and find the +wrong input or nondeterminism, not something to accept automatically. + +## 4. Select validation by change surface + +Run the narrow check while iterating, then the applicable admission checks +before handoff. `make help` is the authoritative target list. + +| Changed surface | Focused check | Admission checks | +|---|---|---| +| Documentation only | inspect links/examples | `git diff --check` | +| Go implementation | `go test ./path/to/package` | `make format-check`, `make test` | +| CLI paths or flags | focused command/help tests | `make build`, `./scripts/policy/check-command-surface.sh --strict`, `make interface-integrity` | +| Schema registry, hints, or generators | focused generator/app tests | `make generate-schema`, `./scripts/policy/check-generated-drift.sh`, `./scripts/policy/check-schema-catalog.sh`, `make test-schema-agent-examples` | +| Skill command examples | inspect referenced `dws` help | `make skill-command-integrity` | +| CI or test sharding | run the affected script/test | `make test-plan`, `make lint`, and the CI workflow's pinned actionlint command when workflows change | +| Packaging or installers | focused release-script tests | `make package`, `./scripts/release/verify-package-managers.sh` | +| Authentication, transport, or OS-specific code | focused tests, including failure paths | `make test`; run relevant platform checks or disclose the unavailable platform | + +`make policy` is the combined policy gate and is appropriate for command, +Schema, generated-asset, or broad cross-cutting changes. Platform credentials +and live services are not prerequisites for ordinary unit tests; never turn a +missing credential into permission to skip deterministic checks. + +The guide contract itself is executable. Run `make coding-agent-harness` after +changing `AGENTS.md`, this guide, the Schema contributor guide, or their routed +commands and paths. This remains a local, opt-in check and is not wired into CI. + +## Design references + +This repository adapts two patterns without copying their product-specific +rules: + +- [Lark CLI's contributor guide](https://github.com/larksuite/cli/blob/5efaf65aec59c33899475bb90e6bff1bc3b5b65c/AGENTS.md): one primary goal, machine-consumable errors/output, and validation selected by behavior surface. +- [WeCom CLI's root routing guide](https://github.com/WecomTeam/wecom-cli/blob/9eb7898b959861af879495e211e37431fa908f19/AGENTS.md) and [human helper template](https://github.com/WecomTeam/wecom-cli/blob/9eb7898b959861af879495e211e37431fa908f19/src/helpers/HUMANS.md): a thin root guide, scoped implementation guidance, and a copyable request format. + +## 5. Pre-handoff self-check + +Confirm every applicable item: + +- The diff implements the stated goal and no unrelated cleanup. +- Pre-existing changes are still present and were not attributed to this task. +- New behavior has a regression test; removed behavior has an explicit reason. +- Public command paths, flags, output, exit behavior, and compatibility remain + intentional. +- Destructive or mutating operations retain the required confirmation path. +- Generated files came from their reviewed inputs and generation is clean. +- Docs and examples use commands accepted by current help/Schema. +- Errors preserve actionable context without leaking secrets. +- `git diff --check` passes and the final diff has been read. +- Every reported check is labeled passed, failed, or not run with a reason. + +Use this compact handoff shape: + +```text +Outcome: what is now true +Files: intentional files changed +Validation: exact commands and results +Limits: unrun checks, environment constraints, follow-ups +``` diff --git a/docs/coding-agent-task-template.md b/docs/coding-agent-task-template.md new file mode 100644 index 00000000..5a86fc24 --- /dev/null +++ b/docs/coding-agent-task-template.md @@ -0,0 +1,35 @@ +# Coding Agent Task Template + +Copy this block into an issue or coding-agent request. One task should have one +primary outcome; split unrelated outcomes instead of hiding them in acceptance +criteria. + +```text +Task kind: bug | feature | refactor | docs | policy | release +Goal (one primary outcome): +Current behavior and evidence: +Acceptance criteria: +In scope (packages/files/surfaces): +Out of scope: +Compatibility constraints: +Interface impact (commands/flags/output/errors/exit codes/Schema): +Safety or data-mutation constraints: +Expected validation: +Known environment limitations: +``` + +For a command or remote-interface task, add the smallest concrete invocation +and contract evidence available: + +```text +CLI path and example argv: +Current --help or Schema excerpt: +Remote method and request/response shape, if relevant: +Expected stdout/stderr and exit behavior: +Mutation preview/confirmation behavior: +``` + +Do not paste credentials, tokens, private endpoints, or production business +data. Use redacted fixtures and say which evidence is unavailable. Save the +filled block and run `make coding-agent-task TASK=path/to/task.md` to validate +it before implementation. diff --git a/docs/schema-contributor-guide.md b/docs/schema-contributor-guide.md new file mode 100644 index 00000000..62298229 --- /dev/null +++ b/docs/schema-contributor-guide.md @@ -0,0 +1,150 @@ +# Schema and Agent Contract Contributor Guide + +Read this guide when changing public CLI commands, Schema identity or +parameters, Agent selection/safety metadata, or generated Schema assets. + +## Ownership and data flow + +The publication graph is one way: + +```text +Cobra command tree + + reviewed CommandRegistry identity/navigation + + reviewed metadata parameter overlays + -> EffectiveCommandRegistry and executable binding + + parameter bindings + + reviewed selection and safety/interface metadata + + pinned MCP metadata + -> one resolved ToolSpec registry/index + -> generated Agent metadata and embedded Schema Catalog + -> dws schema projections and runtime metadata lookup +``` + +The owning sources are: + +| Concern | Authoritative input | +|---|---| +| Executable paths and accepted flags | Cobra tree built by `app.NewRootCommand()` | +| Stable canonical identity, primary path, aliases, navigation | `internal/cli/schema_command_registry.json` | +| Registry editing contract | `internal/cli/schema_command_registry.schema.json` | +| Safety, interface, runtime gates, parameter overlays | `internal/cli/schema_hints/metadata/.json` | +| Agent selection prose and examples | `internal/cli/schema_hints/selection/.json` | +| Product-to-hint-file routing | `internal/cli/schema_hints/index.json` | +| Flag/property bindings | `internal/cli/schema_parameter_bindings.json` | +| Sanitized interface fallback | `internal/cli/schema_mcp_metadata.json` | +| Exact reviewed omissions from Schema | `internal/cli/schema_command_exclusions.json` | + +Generated files under `internal/cli/schema_agent_metadata/` and +`internal/cli/schema_catalog.json` are output only. Runtime loading is a delivery +boundary: it must not create or repair commands, flags, registry entries, or +generation inputs. + +## Invariants + +1. Every delivered tool resolves to a public runnable Cobra leaf. +2. Every public runnable Cobra leaf resolves to Schema or has one exact, + reviewed exclusion with a non-empty reason. Wildcard/prefix exclusions are + forbidden. +3. The reviewed CommandRegistry is the only stable identity/navigation source. + Native annotations, when present, are consistency assertions and must agree. +4. Metadata overlays may describe or constrain real flags; they cannot create + commands, flags, interfaces, or unknown RPCs. +5. Cobra-required flags are a hard floor. An overlay may make an optional flag + required but cannot make a Cobra-required flag optional. +6. Each tool is resolved once into one typed `ToolSpec`; all Catalog, `schema + --all`, leaf, summary, safety, and runtime projections derive from it. +7. Provenance winner values must equal delivered values. Same-precedence + conflicts fail instead of being merged silently. +8. `confirmation=user_required` requires user confirmation before `--yes`. + Do not infer confirmation mechanically from risk/effect; keep runtime gates + and published metadata consistent. +9. Stored examples use real primary/alias paths and accepted flags, satisfy all + required/constraint rules, contain no shell comments, and never add `--yes`. +10. `schema --all` remains the complete compatibility export. Routine discovery + should use overview, product/group, then leaf queries. + +When Help and shipped Schema disagree, treat it as contract drift. Cobra still +defines executable flags; use the safer interpretation for confirmation or +stop rather than guessing. + +Parameter and safety resolution is source-precedence based and otherwise +value-neutral: a value must not win merely because it looks stricter. Preserve +all candidates and the selected source, and fail same-precedence conflicts. +Command text resolves from reviewed tool hints, then command-specific Cobra +help, then MCP metadata; generic RPC prose must not replace a specialized +leaf's description. An alias lookup may change only view fields such as +`cli_path` and `is_alias`, never the resolved command contract. + +## Editing workflow + +1. Confirm the live path and flags in the Cobra tree and current `--help`. +2. Change only the owning reviewed block. Do not copy generated Catalog fields + into inputs or mix selection fields into metadata files. +3. Keep registry edits limited to intentional identity/navigation changes. +4. For selection prose, write decision-oriented routing: when to choose the + command, when a sibling is better, and the result shape. Do not restate help. +5. For parameter overlays, use an exact runnable leaf and real flags; set + `reviewed: true` with a concrete review reason. +6. Regenerate the complete snapshot; publication is deterministic even when + only one product input changed. +7. Inspect authored and generated diffs separately, then run the gates below. + +Pinned MCP metadata is a sanitized fallback. When a task requires refreshing +it and a personal session is available, inspect live metadata with `dws auth +status`, `dws cache refresh`, and `dws schema -f json`. Never print +or commit tokens. Evidence precedence is Runtime/Cobra, live MCP, pinned MCP, +then Skill prose as evidence only. + +## Required checks + +```bash +make generate-schema +./scripts/policy/check-runtime-confirmation-truth.sh +./scripts/policy/check-generated-drift.sh +./scripts/policy/check-schema-catalog.sh +./scripts/policy/check-command-surface.sh --strict +make test-schema-agent-examples +``` + +Also run focused tests for the changed binder, generator, command, or runtime +consumer. Run reverse-completeness tests whenever the Cobra tree changes. + +Agent examples are contract-checked by default. Eligible reviewed dry-run +examples are additionally exercised by `make test-schema-agent-examples` with +isolated state; a runtime failure must not be converted into an ad hoc skip. +Live-model selection evaluation is optional and never a normal CI dependency. + +An example enters runtime dry-run only when its final typed contract publishes +an explicit reviewed dry-run capability. Risk or confirmation metadata does +not manufacture preview support, and the harness never injects `--yes`. A +narrow precondition that cannot be derived from the contract may use an exact, +reviewed `example_dispositions` entry to narrow dry-run to contract-only; it +must not become a general skip or a fallback applied after execution fails. + +Every `use_when` entry is a positive selection fixture and every `avoid_when` +entry is a negative fixture for that tool. The deterministic gate checks +coverage and contradictions; it does not claim to prove natural-language +understanding. When explicitly requested, the optional live-model smoke test +can be run with: + +```bash +DWS_AGENT_SELECTION_LIVE=1 \ +ARK_API_KEY=... ARK_BASE_URL=... ARK_MODEL=... \ +go test ./internal/app -run TestManualAgentSelectionArkLive -count=1 +``` + +Use `DWS_AGENT_SELECTION_FULL=1` for the full fixture or +`DWS_AGENT_SELECTION_CASES=` for selected cases. A custom +HTTPS provider must be explicitly allowlisted; plaintext is accepted only for +a loopback test server so credentials are not sent over arbitrary clear text. + +## Runtime boundaries + +- `schema list` is a progressive overview; `schema --all` is the complete, + non-compact compatibility baseline with full parameters, constraints, and + safety semantics. +- `--compact` saves discovery context but is not a full compatibility export. +- `dws --help` decides whether a path and its flags are executable. Leaf + Schema owns Agent selection, mapping, constraints, and safety semantics. +- Help and Schema describe commands; they do not return DingTalk business + data. Execute the real read/list/search command after discovery. diff --git a/internal/cli/AGENTS.md b/internal/cli/AGENTS.md new file mode 100644 index 00000000..8e7538ea --- /dev/null +++ b/internal/cli/AGENTS.md @@ -0,0 +1,42 @@ +# Schema Runtime and Publication Agent Guide + +This file applies to `internal/cli/`. Read +[`docs/schema-contributor-guide.md`](../../docs/schema-contributor-guide.md) +before editing. + +## Owning inputs + +| Change | Edit | +|---|---| +| Canonical identity, primary path, aliases, navigation | `schema_command_registry.json` | +| Parameter/property mapping | `schema_parameter_bindings.json` or reviewed metadata overlay | +| Safety, interface, runtime gate | `schema_hints/metadata/.json` | +| Agent selection and examples | `schema_hints/selection/.json` | +| Exact reviewed omission | `schema_command_exclusions.json` | +| Runtime query/projection behavior | Go implementation and focused tests in this package | + +The executable Cobra tree outside this package owns whether a command exists +and which flags it accepts. Do not create a command or flag in Schema inputs. + +## Generated boundary + +- `schema_catalog.json` and `schema_agent_metadata/` are generated outputs. +- Never hand-edit, merge from, or use a previous generated output as an input. +- Change the owning reviewed input or generator, run `make generate-schema`, + and inspect authored and generated diffs separately. +- A broad unrelated generated diff is a failure signal, not acceptable churn. + +## Self-check + +- Every public runnable Cobra leaf is bound or has one exact reviewed + exclusion; every delivered tool binds back to a runnable leaf. +- Registry identity and native annotations agree; aliases do not mutate the + resolved contract. +- Parameters reference real flags and retain Cobra-required floors. +- Selection examples use executable paths/flags and never include `--yes`. +- Runtime confirmation gates and published safety metadata agree. +- Full, compact, summary, alias, and Catalog projections derive from the same + typed `ToolSpec` and preserve provenance winners. + +Run the focused package/generator tests and the complete command list in +`docs/schema-contributor-guide.md`, beginning with `make generate-schema`. diff --git a/internal/helpers/AGENTS.md b/internal/helpers/AGENTS.md new file mode 100644 index 00000000..bca32567 --- /dev/null +++ b/internal/helpers/AGENTS.md @@ -0,0 +1,56 @@ +# Product Command Handler Agent Guide + +This file applies to `internal/helpers/`. Read the root `AGENTS.md`, +[`CONTRIBUTING.md`](../../CONTRIBUTING.md), and +[`docs/coding-agent-guide.md`](../../docs/coding-agent-guide.md) first. + +## Scope and routing + +`internal/helpers` owns product command construction and handler behavior. Find +the owning product file and its closest tests before editing. + +| Concern | Owning surface | +|---|---| +| Root/static command wiring or plugin loading | `internal/app` | +| Product command flags and handler behavior | `internal/helpers` | +| Shared Cobra construction | `internal/cobracmd` | +| Invocation and transport | `internal/executor`, `internal/transport` | +| Structured failures and recovery hints | `internal/errors`, `internal/recovery` | +| Output encoding and projections | `internal/output` | +| Confirmation and dry-run guards | `internal/safety` and command runtime gates | +| Agent identity/Schema/parameters | `internal/cli` and [`docs/schema-contributor-guide.md`](../../docs/schema-contributor-guide.md) | + +Do not modify `internal/app` or shared layers merely to register an ordinary +product leaf when the existing helper construction already owns it. Cross the +package boundary only when the task changes that shared contract. + +## Command contract + +- Confirm the current path and flags with `dws --help` and the Cobra + construction before changing behavior. +- Keep stdout machine-readable business data. Send progress, warnings, and + diagnostics through the established stderr/logging paths. +- Preserve structured error category, stable exit behavior, cause, trace ID, + and an actionable recovery hint. Do not replace a typed lower-layer failure + with an unclassified message. +- Treat paths, JSON payloads, filenames, and remote content as untrusted input; + use existing validation and sanitization helpers. +- Mutating or destructive commands must keep their preview/confirmation + behavior aligned with runtime safety and published Schema metadata. +- If flags, identity, parameters, selection, or safety metadata change, follow + the scoped `internal/cli/AGENTS.md` and regenerate from reviewed inputs. + +## Verification matrix + +| Change | Required focused evidence | +|---|---| +| Handler bug or behavior | package regression test including the failure path | +| Flags or request mapping | Cobra/help test plus dry-run request assertion when the command supports preview | +| Output or error behavior | structured payload, stderr/stdout, and exit-code assertions | +| Mutating behavior | preview/confirmation test; live execution only with explicit authorization and disposable data | +| Shared helper refactor | affected product tests plus `go test ./internal/helpers/...` | +| Public command surface | Schema/command gates from `internal/cli/AGENTS.md` | + +Before handoff, run the narrow package tests, `make format-check`, and the +admission checks selected by `docs/coding-agent-guide.md`. Do not claim a live +round-trip when only dry-run or mocked transport was exercised. diff --git a/scripts/policy/check-coding-agent-harness.sh b/scripts/policy/check-coding-agent-harness.sh new file mode 100755 index 00000000..ac3cfeb0 --- /dev/null +++ b/scripts/policy/check-coding-agent-harness.sh @@ -0,0 +1,7 @@ +#!/bin/sh +set -eu + +ROOT="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)" + +cd "$ROOT" +exec go run ./scripts/policy/coding-agent-harness -root "$ROOT" "$@" diff --git a/scripts/policy/coding-agent-harness/main.go b/scripts/policy/coding-agent-harness/main.go new file mode 100644 index 00000000..5bdeb44d --- /dev/null +++ b/scripts/policy/coding-agent-harness/main.go @@ -0,0 +1,403 @@ +package main + +import ( + "bufio" + "errors" + "flag" + "fmt" + "os" + "path/filepath" + "regexp" + "sort" + "strings" +) + +var guideLineLimits = map[string]int{ + "AGENTS.md": 80, + "internal/cli/AGENTS.md": 80, + "internal/helpers/AGENTS.md": 100, +} + +var markdownLinkPattern = regexp.MustCompile(`\[[^]]+\]\(([^)]+)\)`) + +type fileContract struct { + path string + required []string +} + +type taskField struct { + label string + allowNone bool + allowedValue map[string]bool +} + +var taskFields = []taskField{ + {label: "Task kind", allowedValue: map[string]bool{"bug": true, "feature": true, "refactor": true, "docs": true, "policy": true, "release": true}}, + {label: "Goal (one primary outcome)"}, + {label: "Current behavior and evidence"}, + {label: "Acceptance criteria"}, + {label: "In scope (packages/files/surfaces)"}, + {label: "Out of scope", allowNone: true}, + {label: "Compatibility constraints", allowNone: true}, + {label: "Interface impact (commands/flags/output/errors/exit codes/Schema)", allowNone: true}, + {label: "Safety or data-mutation constraints", allowNone: true}, + {label: "Expected validation"}, + {label: "Known environment limitations", allowNone: true}, +} + +var guideContracts = []fileContract{ + { + path: "AGENTS.md", + required: []string{ + "docs/coding-agent-guide.md", + "docs/coding-agent-task-template.md", + "docs/schema-contributor-guide.md", + "internal/helpers/AGENTS.md", + "docs/automation.md", + "docs/agent-code.md", + "Do not depend on generated Wiki or CodeWiki content.", + }, + }, + { + path: "CONTRIBUTING.md", + required: []string{ + "docs/coding-agent-guide.md", + "docs/schema-contributor-guide.md", + }, + }, + { + path: "docs/coding-agent-guide.md", + required: []string{ + "## 1. Normalize the task input", + "coding-agent-task-template.md", + "make coding-agent-task TASK=", + "primary outcome per task", + "github.com/larksuite/cli/blob/", + "github.com/WecomTeam/wecom-cli/blob/", + "## 2. Establish the baseline", + "## 3. Implement from authoritative inputs", + "## 4. Select validation by change surface", + "Documentation only", + "Go implementation", + "CLI paths or flags", + "Schema registry, hints, or generators", + "CI or test sharding", + "Packaging or installers", + "Authentication, transport, or OS-specific code", + "## 5. Pre-handoff self-check", + "Outcome: what is now true", + "Validation: exact commands and results", + }, + }, + { + path: "docs/coding-agent-task-template.md", + required: []string{ + "Goal (one primary outcome):", + "Current behavior and evidence:", + "Acceptance criteria:", + "In scope (packages/files/surfaces):", + "Out of scope:", + "Compatibility constraints:", + "Interface impact (commands/flags/output/errors/exit codes/Schema):", + "Safety or data-mutation constraints:", + "Expected validation:", + "Known environment limitations:", + "Expected stdout/stderr and exit behavior:", + "Mutation preview/confirmation behavior:", + }, + }, + { + path: "docs/schema-contributor-guide.md", + required: []string{ + "internal/cli/schema_command_registry.json", + "internal/cli/schema_command_registry.schema.json", + "internal/cli/schema_hints/metadata/.json", + "internal/cli/schema_hints/selection/.json", + "internal/cli/schema_parameter_bindings.json", + "internal/cli/schema_mcp_metadata.json", + "internal/cli/schema_command_exclusions.json", + "internal/cli/schema_catalog.json", + "make generate-schema", + "make test-schema-agent-examples", + }, + }, + { + path: "internal/helpers/AGENTS.md", + required: []string{ + "../../CONTRIBUTING.md", + "../../docs/coding-agent-guide.md", + "../../docs/schema-contributor-guide.md", + "Keep stdout machine-readable business data.", + "structured error category", + "preview/confirmation", + "Do not claim a live", + }, + }, + { + path: "internal/cli/AGENTS.md", + required: []string{ + "../../docs/schema-contributor-guide.md", + "schema_command_registry.json", + "schema_parameter_bindings.json", + "schema_hints/metadata/.json", + "schema_hints/selection/.json", + "schema_command_exclusions.json", + "schema_catalog.json", + "make generate-schema", + }, + }, +} + +var requiredPaths = []string{ + "docs/automation.md", + "docs/agent-code.md", + "scripts/policy/check-runtime-confirmation-truth.sh", + "scripts/policy/check-coding-agent-harness.sh", + "scripts/policy/check-generated-drift.sh", + "scripts/policy/check-schema-catalog.sh", + "scripts/policy/check-command-surface.sh", + "scripts/release/verify-package-managers.sh", +} + +var requiredMakeTargets = []string{ + "build", + "coding-agent-harness", + "coding-agent-task", + "format-check", + "test", + "policy", + "interface-integrity", + "skill-command-integrity", + "test-schema-agent-examples", + "generate-schema", + "package", +} + +func main() { + root := flag.String("root", ".", "repository root") + task := flag.String("task", "", "optional filled coding-agent task file to validate") + flag.Parse() + if flag.NArg() != 0 { + fmt.Fprintln(os.Stderr, "coding agent harness: unexpected positional arguments") + os.Exit(2) + } + + problems := validate(*root) + if strings.TrimSpace(*task) != "" { + taskPath := *task + if !filepath.IsAbs(taskPath) { + taskPath = filepath.Join(*root, taskPath) + } + problems = append(problems, validateTask(taskPath)...) + sort.Slice(problems, func(i, j int) bool { return problems[i].Error() < problems[j].Error() }) + } + if len(problems) != 0 { + fmt.Fprintln(os.Stderr, "coding agent harness: failed") + for _, problem := range problems { + fmt.Fprintf(os.Stderr, "- %v\n", problem) + } + os.Exit(1) + } + fmt.Println("coding agent harness: ok") +} + +func validateTask(path string) []error { + content, err := os.ReadFile(path) + if err != nil { + return []error{fmt.Errorf("read task contract %s: %w", path, err)} + } + + values := make(map[string][]string) + occurrences := make(map[string]int) + current := "" + scanner := bufio.NewScanner(strings.NewReader(string(content))) + for scanner.Scan() { + line := strings.TrimSpace(scanner.Text()) + matched := false + for _, field := range taskFields { + prefix := field.label + ":" + if strings.HasPrefix(line, prefix) { + current = field.label + occurrences[current]++ + inline := strings.TrimSpace(strings.TrimPrefix(line, prefix)) + if inline != "" { + values[current] = append(values[current], inline) + } + matched = true + break + } + } + if matched || current == "" || line == "" || strings.HasPrefix(line, "```") { + continue + } + values[current] = append(values[current], line) + } + if err := scanner.Err(); err != nil { + return []error{fmt.Errorf("scan task contract %s: %w", path, err)} + } + + var problems []error + for _, field := range taskFields { + if occurrences[field.label] > 1 { + problems = append(problems, fmt.Errorf("task contract field %q appears %d times; each field must be unique", field.label, occurrences[field.label])) + } + value := strings.TrimSpace(strings.Join(values[field.label], "\n")) + if value == "" { + problems = append(problems, fmt.Errorf("task contract is missing a value for %q", field.label)) + continue + } + normalized := strings.ToLower(value) + if len(field.allowedValue) != 0 && !field.allowedValue[normalized] { + problems = append(problems, fmt.Errorf("task contract field %q has unsupported value %q", field.label, value)) + } + if isPlaceholderTaskValue(normalized) { + problems = append(problems, fmt.Errorf("task contract field %q still contains placeholder value %q", field.label, value)) + } + if !field.allowNone && (normalized == "none" || normalized == "n/a" || normalized == "not applicable") { + problems = append(problems, fmt.Errorf("task contract field %q requires concrete evidence", field.label)) + } + } + sort.Slice(problems, func(i, j int) bool { return problems[i].Error() < problems[j].Error() }) + return problems +} + +func isPlaceholderTaskValue(value string) bool { + for _, line := range strings.Split(value, "\n") { + trimmed := strings.TrimSpace(line) + trimmed = strings.TrimLeft(trimmed, "-*0123456789.) ") + switch trimmed { + case "todo", "tbd", "unknown", "fill me", "to be decided": + return true + } + if strings.HasPrefix(trimmed, "todo:") || strings.HasPrefix(trimmed, "tbd:") || strings.Contains(trimmed, " limit { + problems = append(problems, fmt.Errorf("%s has %d lines; scoped guide limit is %d", path, lineCount, limit)) + } + } + + for _, path := range requiredPaths { + info, statErr := os.Stat(filepath.Join(absRoot, filepath.FromSlash(path))) + if statErr != nil { + problems = append(problems, fmt.Errorf("required referenced path %s: %w", path, statErr)) + } else if strings.HasSuffix(path, ".sh") && info.Mode().Perm()&0o111 == 0 { + problems = append(problems, fmt.Errorf("required referenced script %s is not executable", path)) + } + } + + makeTargets, makeErr := loadMakeTargets(filepath.Join(absRoot, "Makefile")) + if makeErr != nil { + problems = append(problems, makeErr) + } else { + for _, target := range requiredMakeTargets { + if !makeTargets[target] { + problems = append(problems, fmt.Errorf("makefile is missing referenced target %q", target)) + } + } + } + + sort.Slice(problems, func(i, j int) bool { return problems[i].Error() < problems[j].Error() }) + return problems +} + +func validateLocalLinks(root, document, content string) []error { + var problems []error + documentDir := filepath.Join(root, filepath.Dir(filepath.FromSlash(document))) + for _, match := range markdownLinkPattern.FindAllStringSubmatch(content, -1) { + target := strings.TrimSpace(match[1]) + if target == "" || strings.HasPrefix(target, "#") || strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") || strings.HasPrefix(target, "mailto:") { + continue + } + if cut, _, ok := strings.Cut(target, "#"); ok { + target = cut + } + if target == "" { + continue + } + if filepath.IsAbs(target) { + problems = append(problems, fmt.Errorf("%s contains absolute local link %q", document, match[1])) + continue + } + resolved := filepath.Clean(filepath.Join(documentDir, filepath.FromSlash(target))) + rel, err := filepath.Rel(root, resolved) + if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { + problems = append(problems, fmt.Errorf("%s link %q escapes repository root", document, match[1])) + continue + } + if _, err := os.Stat(resolved); err != nil { + problems = append(problems, fmt.Errorf("%s has broken local link %q: %w", document, match[1], err)) + } + } + return problems +} + +func countLines(path string) (int, error) { + file, err := os.Open(path) + if err != nil { + return 0, err + } + defer file.Close() + + scanner := bufio.NewScanner(file) + count := 0 + for scanner.Scan() { + count++ + } + return count, scanner.Err() +} + +func loadMakeTargets(path string) (map[string]bool, error) { + file, err := os.Open(path) + if err != nil { + return nil, fmt.Errorf("read Makefile targets: %w", err) + } + defer file.Close() + + targets := make(map[string]bool) + scanner := bufio.NewScanner(file) + for scanner.Scan() { + line := scanner.Text() + if line == "" || line[0] == '\t' || strings.HasPrefix(strings.TrimSpace(line), "#") { + continue + } + name, _, ok := strings.Cut(line, ":") + if !ok || strings.ContainsAny(name, "= ") || strings.HasPrefix(name, ".") { + continue + } + targets[name] = true + } + if err := scanner.Err(); err != nil { + return nil, errors.New("scan Makefile targets: " + err.Error()) + } + return targets, nil +} diff --git a/scripts/policy/coding-agent-harness/main_test.go b/scripts/policy/coding-agent-harness/main_test.go new file mode 100644 index 00000000..10b378fc --- /dev/null +++ b/scripts/policy/coding-agent-harness/main_test.go @@ -0,0 +1,250 @@ +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" +) + +func TestRepositoryCodingAgentHarness(t *testing.T) { + root, err := filepath.Abs(filepath.Join("..", "..", "..")) + if err != nil { + t.Fatalf("resolve repository root: %v", err) + } + if problems := validate(root); len(problems) != 0 { + t.Fatalf("repository harness problems:\n%s", formatProblems(problems)) + } +} + +func TestCodingAgentHarnessFixturePasses(t *testing.T) { + root := newHarnessFixture(t) + if problems := validate(root); len(problems) != 0 { + t.Fatalf("valid fixture problems:\n%s", formatProblems(problems)) + } +} + +func TestCodingAgentTaskContractPasses(t *testing.T) { + path := filepath.Join(t.TempDir(), "task.md") + writeTaskFixture(t, path, validTaskContract) + if problems := validateTask(path); len(problems) != 0 { + t.Fatalf("valid task problems:\n%s", formatProblems(problems)) + } +} + +func TestCodingAgentTaskContractFailsClosed(t *testing.T) { + tests := []struct { + name string + content string + wantProblem string + }{ + { + name: "missing goal", + content: strings.Replace(validTaskContract, "Goal (one primary outcome): Fix retry handling\n", "", 1), + wantProblem: `missing a value for "Goal (one primary outcome)"`, + }, + { + name: "blank acceptance criteria", + content: strings.Replace(validTaskContract, "Acceptance criteria:\n- The retry succeeds once and then stops.\n", "Acceptance criteria:\n", 1), + wantProblem: `missing a value for "Acceptance criteria"`, + }, + { + name: "unsupported task kind", + content: strings.Replace(validTaskContract, "Task kind: bug", "Task kind: experiment", 1), + wantProblem: `unsupported value "experiment"`, + }, + { + name: "placeholder acceptance bullet", + content: strings.Replace(validTaskContract, "- The retry succeeds once and then stops.", "- TBD: decide expected retry result", 1), + wantProblem: "still contains placeholder value", + }, + { + name: "duplicate goal", + content: strings.Replace(validTaskContract, "Goal (one primary outcome): Fix retry handling", "Goal (one primary outcome): Fix retry handling\nGoal (one primary outcome): Refactor transport", 1), + wantProblem: `field "Goal (one primary outcome)" appears 2 times`, + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + path := filepath.Join(t.TempDir(), "task.md") + writeTaskFixture(t, path, test.content) + problems := formatProblems(validateTask(path)) + if !strings.Contains(problems, test.wantProblem) { + t.Fatalf("problems:\n%s\nwant substring %q", problems, test.wantProblem) + } + }) + } +} + +func TestCodingAgentHarnessFailsClosed(t *testing.T) { + tests := []struct { + name string + mutate func(t *testing.T, root string) + wantProblem string + }{ + { + name: "root guide is no longer thin", + mutate: func(t *testing.T, root string) { + path := filepath.Join(root, "AGENTS.md") + file, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + defer file.Close() + for i := 0; i < guideLineLimits["AGENTS.md"]; i++ { + if _, err := file.WriteString("extra line\n"); err != nil { + t.Fatal(err) + } + } + }, + wantProblem: "scoped guide limit", + }, + { + name: "task input field is removed", + mutate: func(t *testing.T, root string) { + replaceFixtureText(t, root, "docs/coding-agent-task-template.md", "Acceptance criteria:", "") + }, + wantProblem: `docs/coding-agent-task-template.md is missing required contract text "Acceptance criteria:"`, + }, + { + name: "route link is broken", + mutate: func(t *testing.T, root string) { + replaceFixtureText(t, root, "AGENTS.md", "[automation](docs/automation.md)", "[automation](docs/missing.md)") + }, + wantProblem: "broken local link", + }, + { + name: "referenced script is absent", + mutate: func(t *testing.T, root string) { + if err := os.Remove(filepath.Join(root, "scripts/policy/check-schema-catalog.sh")); err != nil { + t.Fatal(err) + } + }, + wantProblem: "required referenced path scripts/policy/check-schema-catalog.sh", + }, + { + name: "make target is absent", + mutate: func(t *testing.T, root string) { + replaceFixtureText(t, root, "Makefile", "package:\n", "removed-package:\n") + }, + wantProblem: `makefile is missing referenced target "package"`, + }, + { + name: "harness script is not executable", + mutate: func(t *testing.T, root string) { + path := filepath.Join(root, "scripts/policy/check-coding-agent-harness.sh") + if err := os.Chmod(path, 0o644); err != nil { + t.Fatal(err) + } + }, + wantProblem: "required referenced script scripts/policy/check-coding-agent-harness.sh is not executable", + }, + } + + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + root := newHarnessFixture(t) + test.mutate(t, root) + problems := formatProblems(validate(root)) + if !strings.Contains(problems, test.wantProblem) { + t.Fatalf("problems:\n%s\nwant substring %q", problems, test.wantProblem) + } + }) + } +} + +func newHarnessFixture(t *testing.T) string { + t.Helper() + root := t.TempDir() + + for _, contract := range guideContracts { + var content strings.Builder + content.WriteString("# Fixture\n\n") + for _, required := range contract.required { + content.WriteString(required) + content.WriteByte('\n') + } + if contract.path == "AGENTS.md" { + content.WriteString("[coding](docs/coding-agent-guide.md)\n") + content.WriteString("[schema](docs/schema-contributor-guide.md)\n") + content.WriteString("[automation](docs/automation.md)\n") + content.WriteString("[agent code](docs/agent-code.md)\n") + } + writeFixtureFile(t, root, contract.path, content.String()) + } + + for _, path := range requiredPaths { + writeFixtureFileMode(t, root, path, "fixture\n", 0o755) + } + var makefile strings.Builder + for _, target := range requiredMakeTargets { + makefile.WriteString(target) + makefile.WriteString(":\n\t@true\n") + } + writeFixtureFile(t, root, "Makefile", makefile.String()) + return root +} + +func writeFixtureFile(t *testing.T, root, path, content string) { + t.Helper() + writeFixtureFileMode(t, root, path, content, 0o644) +} + +func writeFixtureFileMode(t *testing.T, root, path, content string, mode os.FileMode) { + t.Helper() + full := filepath.Join(root, filepath.FromSlash(path)) + if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil { + t.Fatalf("create fixture directory: %v", err) + } + if err := os.WriteFile(full, []byte(content), mode); err != nil { + t.Fatalf("write fixture %s: %v", path, err) + } +} + +func replaceFixtureText(t *testing.T, root, path, old, replacement string) { + t.Helper() + full := filepath.Join(root, filepath.FromSlash(path)) + content, err := os.ReadFile(full) + if err != nil { + t.Fatalf("read fixture %s: %v", path, err) + } + updated := strings.Replace(string(content), old, replacement, 1) + if updated == string(content) { + t.Fatalf("fixture %s does not contain %q", path, old) + } + if err := os.WriteFile(full, []byte(updated), 0o644); err != nil { + t.Fatalf("update fixture %s: %v", path, err) + } +} + +func formatProblems(problems []error) string { + var lines []string + for _, problem := range problems { + lines = append(lines, problem.Error()) + } + return strings.Join(lines, "\n") +} + +func writeTaskFixture(t *testing.T, path, content string) { + t.Helper() + if err := os.WriteFile(path, []byte(content), 0o644); err != nil { + t.Fatalf("write task fixture: %v", err) + } +} + +const validTaskContract = `# Task + +Task kind: bug +Goal (one primary outcome): Fix retry handling +Current behavior and evidence: The focused test reproduces two retries. +Acceptance criteria: +- The retry succeeds once and then stops. +In scope (packages/files/surfaces): internal/transport +Out of scope: Authentication behavior +Compatibility constraints: Preserve existing flags and output +Interface impact (commands/flags/output/errors/exit codes/Schema): None +Safety or data-mutation constraints: No external writes +Expected validation: Focused unit test and make test +Known environment limitations: Live service unavailable +` diff --git a/scripts/policy/coding-agent-harness/testdata/valid-task.md b/scripts/policy/coding-agent-harness/testdata/valid-task.md new file mode 100644 index 00000000..7ef62f4a --- /dev/null +++ b/scripts/policy/coding-agent-harness/testdata/valid-task.md @@ -0,0 +1,15 @@ +# Retry Handling Task + +Task kind: bug +Goal (one primary outcome): Fix transient retry handling +Current behavior and evidence: A focused transport test reproduces two retries after a successful response. +Acceptance criteria: +- The retry loop stops after the first successful response. +- The existing output and exit behavior remain unchanged. +In scope (packages/files/surfaces): internal/transport retry implementation and tests +Out of scope: Authentication and endpoint discovery +Compatibility constraints: Preserve existing flags, JSON output, and exit codes +Interface impact (commands/flags/output/errors/exit codes/Schema): None +Safety or data-mutation constraints: No live service calls or external writes +Expected validation: Focused transport tests, race test, formatting, and full Go test suite +Known environment limitations: Live DingTalk service is intentionally not used