feat: add local coding agent harness

This commit is contained in:
玉澜 committed 2026-08-01 13:26:34 +08:00
1 parent 412e77f215
commit 45dc8a439c
12 files changed
+1117 -348

No files matched your search

+35 -344
View File
@@ -1,357 +1,48 @@
# Repository Agent Guide
This file applies to the entire repository. Keep changes scoped, preserve
unrelated work, and use `gofmt` for every modified Go file.
This file applies to the entire repository. Keep it as a routing page: load
the detailed guide for the surface you are changing instead of treating this
file as a repository wiki.
## Build and test
## Always
- Build: `go build ./cmd`
- Full test suite: `DWS_PACKAGE_VERSION=0.0.0-test go test ./...`
- Generate Schema assets: `go generate ./internal/cli`
- Check generated drift: `./scripts/policy/check-generated-drift.sh`
- Check the Schema contract: `./scripts/policy/check-schema-catalog.sh`
- Preserve unrelated and pre-existing work; inspect `git status` before edits.
- Make the smallest coherent change and update its tests and user-facing docs.
- Use `gofmt` for every modified Go file.
- Treat repository code, tests, scripts, and versioned docs as the source of
truth. Do not depend on generated Wiki or CodeWiki content.
- Do not hand-edit generated Schema Catalog or Agent metadata. Change their
reviewed inputs or generators, then regenerate.
Generated Schema JSON is committed. Change its source inputs and generators,
then regenerate; do not hand-edit generated Catalog or Agent metadata files.
`internal/cli/schema_command_registry.json` is different: it is a reviewed
`CommandRegistry` source, not a generated snapshot. It is the single reviewed
source of stable canonical identity,
primary paths, aliases, and navigation. Edit it only when reviewed exposure,
identity, primary path, or aliases change; parameter, Skill, and metadata-only
changes must not rewrite it mechanically.
## Read by task
## Agent Schema contract
| Change surface | Required guide |
|---|---|
| Any implementation or review | [`CONTRIBUTING.md`](CONTRIBUTING.md) and [`docs/coding-agent-guide.md`](docs/coding-agent-guide.md) |
| Writing a task for a coding agent | [`docs/coding-agent-task-template.md`](docs/coding-agent-task-template.md) |
| Product command handler behavior | [`internal/helpers/AGENTS.md`](internal/helpers/AGENTS.md) |
| CLI paths, flags, Schema, Agent metadata, or generated Catalog | [`docs/schema-contributor-guide.md`](docs/schema-contributor-guide.md) |
| CI, release, packaging, or repository automation | [`docs/automation.md`](docs/automation.md) |
| Agent identification headers or host integration | [`docs/agent-code.md`](docs/agent-code.md) |
The Schema data flow is one way:
Read the closest code and tests for the affected package as well. Nested
`AGENTS.md` files take precedence for their subtrees.
```text
1. app.NewRootCommand()
└─ builds the real Cobra command tree and flags
## Common checks
2. schema_command_registry.json
+ schema_hints/metadata/<product>.json tool parameters (+ cli_path)
└─ forms EffectiveCommandRegistry
└─ binds exactly to real Cobra leaves and aliases
3. Parameter resolution
Cobra flags
+ schema_parameter_bindings.json
+ metadata tool parameters
└─ produces ParameterSpec and constraints
4. Agent and interface semantics
schema_hints/selection/<product>.json (selection prose)
+ schema_hints/metadata/<product>.json (safety/interface/runtime_gate)
+ pinned MCP metadata
└─ resolves Agent metadata by source precedence
Markdown is evidence only; it is not concatenated into final prose
5. One typed hub
BoundCommandRegistry
+ ParameterSpec
+ Agent metadata
+ Interface metadata
└─ resolves every command exactly once into ToolSpec
└─ aggregates SchemaRegistry + SchemaIndex
6. One-way publication
SchemaRegistry
└─ internal/cli/schema_catalog.json
└─ dws schema list/product/group/leaf/--all
```
Parameter overlays from metadata are merged into `EffectiveCommandRegistry`
*before* Cobra binding; after that point there is no second identity source and
no identity precedence winner. The binder must reject a missing/non-runnable
Cobra path, an alias collision, and any native identity annotation that
disagrees with the effective registry. A missing native identity annotation is
allowed because annotations are implementation-side assertions, not identity
fallbacks.
The assembler resolves every bound command exactly once into one `ToolSpec`.
Build-time gates and the snapshot serializer consume that source-resolved typed
registry/index. Runtime projections and delivery gates consume the typed
registry/index returned by the production snapshot loader. Neither path may
reopen annotations, merge source records, or use a previous Catalog or other
generated JSON as a source. `schema_catalog.json` is output-only in the
generation graph. The production loader decoding the embedded published
snapshot is a delivery boundary, not source resolution; it must never create or
repair a Cobra command, flag, registry entry, or later Catalog generation.
This split is architecturally isomorphic to Lark's typed metadata registry,
navigation catalog, and schema renderer. DWS intentionally preserves its
existing flat JSON wire contract for compatibility; do not treat architectural
alignment as permission to make an unversioned wire-format change.
The reviewed `CommandRegistry` is the sole source of stable command identity
and navigation. The executable Cobra tree remains the source of truth for
whether a CLI path exists, is runnable, and which flags it accepts. Schema
coverage is bidirectional:
1. Every final `SchemaRegistry` tool, including its serialized Catalog
projection, must resolve to an executable Cobra command.
2. Every public runnable Cobra leaf must either resolve to Schema or appear as
an exact, reviewed exclusion with a non-empty reason in
`internal/cli/schema_command_exclusions.json`.
Do not use prefix or wildcard exclusions: they can silently hide future
commands. Remove an exclusion when its command enters Schema; stale, invalid,
or duplicate exclusions must fail generation and CI.
When adding or changing an Agent-visible command, review all relevant inputs:
- `internal/cli/schema_command_registry.json` for the reviewed
`CommandRegistry`: canonical identity, primary CLI path, aliases, and stable
navigation. It is the identity source and is not a generated artifact.
- `internal/cli/schema_command_registry.schema.json` is its closed,
machine-readable editing contract. Preserve the local `$schema` reference;
unknown fields, invalid visibility values, stale paths, and collisions fail
Go validation and policy.
- `internal/cli/schema_hints/metadata/<product>.json` for safety, interface,
`runtime_gate`, and optional parameter overlays (`parameters` / `cli_path`).
- `internal/cli/schema_hints/selection/<product>.json` for reviewed Agent
selection prose (`agent_summary`, `use_when`, `avoid_when`, `examples`).
- `internal/cli/schema_hints/index.json` only maps product IDs to those files.
- Native Runtime Schema identity annotations, when present, as consistency
assertions against `EffectiveCommandRegistry`. They must agree exactly and
must never materialize, infer, or override registry identity.
- Flag-to-interface property mappings and required/default semantics.
- Generated files under `internal/cli/schema_agent_metadata/` and
`internal/cli/schema_catalog.json` after running generation.
Run the reverse-completeness tests whenever the Cobra tree changes. A command
that works through `dws <path>` but cannot be found through the matching
`dws schema` lookup is a contract failure unless it has a reviewed exact
exclusion.
Metadata parameter overlays must reference an exact public runnable Cobra leaf
and real flags. They may override Schema description, interface-property/type
mapping, `required`, and `required_when`; they must not create commands or
flags, define an interface, or advertise an unknown RPC. Every authored entry
requires `reviewed: true` and a non-empty review reason.
For Agent-authored metadata or selection edits:
1. Confirm the exact command and flag names in the current Cobra tree.
2. Edit only the owning block (`metadata/` or `selection/`); do not mix fields.
3. Add the smallest possible entry; do not copy generated Catalog fields into
the input.
4. Describe user-visible semantics in `review_reason` and parameter
descriptions.
5. Run generation, drift, Schema policy, and the focused CLI tests before
proposing the change.
## Agent curation workflow (Schema hints)
Use this workflow when refreshing Agent selection prose and confirmation
alignment. Prefer **agent-authored review** over bulk merge scripts that dump
`selection-review.json` or Skill Markdown into Catalog fields.
Human-authored inputs are split into two blocks:
| Block | Path | Owns |
|---|---|---|
| **metadata** | `internal/cli/schema_hints/metadata/<product>.json` | `effect` / `risk` / `confirmation` / `idempotency` / `interface_*` / `runtime_gate` / optional `parameters` |
| **selection** | `internal/cli/schema_hints/selection/<product>.json` | `agent_summary` / `use_when` / `avoid_when` / `examples` (+ product routing) |
`index.json` only maps product IDs to those files. Do not mix selection fields
into metadata files or metadata fields into selection files.
### Goals
1. **Selection prose** is decision-oriented (Feishu/Lark style): trigger intent,
sibling-command routing, and outcome shape — not a restatement of the
summary. Delivered Catalog provenance is `reviewed_explicit` from
`selection/`.
2. **Safety** follows Runtime: `confirmation=user_required` iff the tool's
metadata `runtime_gate != none` (for example `confirm_delete`, `typed_yes`,
`confirm_dangerous`).
3. **Parameter overrides** (former Manual `commands`) live on metadata tools as
`parameters` (+ `cli_path`) and are applied into EffectiveCommandRegistry.
### Authoring
For every curated tool:
1. Edit `metadata/<product>.json` for safety/interface/gates/parameters.
2. Edit `selection/<product>.json` for selection prose (`reviewed: true`,
`review_reason`, `source_refs`).
3. Run `make generate-schema`. Do not hand-edit generated
`schema_agent_metadata/` or `schema_catalog.json`.
### Pull live MCP descriptions (personal token)
Pinned `internal/cli/schema_mcp_metadata.json` is a sanitized baseline. Prefer
live Schema from a logged-in personal session:
Choose checks from the matrix in `docs/coding-agent-guide.md`; do not claim a
check that was not run.
```bash
dws auth status # token_valid should be true
dws cache refresh # refresh discovery / tools cache
dws schema <mcp-canonical> -f json
# or CLI path: dws schema --cli-path "drive copy" -f json
make coding-agent-harness
make build
make format-check
make test
make policy
git diff --check
```
Resolve MCP identity via `interface_ref` when CLI canonical ≠ MCP path
(example: CLI `drive.copy_document` → live `doc.copy_document`). On pull
failure, fall back to Skill + Cobra Help + pinned MCP, and record evidence
(for example `live-dws-schema:<path>#FAILED`). Never print or commit tokens.
Precedence when sources disagree: **Runtime/Cobra > live MCP > pinned MCP >
Skill (evidence only)**.
### Parallel product agents
Split work by product groups. Each agent must:
- Read Skill, Cobra/`--help`, Runtime confirmation sites, and live `dws schema`
for its tools.
- Hand-write selection + metadata; forbid wholesale JSON merges from review
dumps.
- Edit only its `metadata/<product>.json` and `selection/<product>.json`.
- **Never** `git checkout` unrelated product files to “clean scope”.
### Regenerate and gates
```bash
make generate-schema
./scripts/policy/check-runtime-confirmation-truth.sh
go test ./internal/app -run '^TestSheetFinalSchemaConfirmationMatchesRuntimeGuards$' -count=1
```
Example rules (fail generation otherwise):
- At most two examples per tool; no `--yes` in stored examples.
- Examples must match live Cobra argv (path, flags, required groups).
- No shell comments in examples.
After generation, spot-check Catalog: selection provenance is
`reviewed_explicit` from `selection/`, and `user_required` count equals
metadata `runtime_gate != none`.
`make generate-schema` is a full deterministic snapshot rebuild, not an
incremental patch over the previous Catalog. It rereads every reviewed input,
removes stale generated product metadata, and rewrites the exact metadata and
Catalog projections. Incremental work happens only when an Agent or human
edits selected `metadata/` or `selection/` entries; the next publication still
recomputes all outputs. Generated files must never be read back as merge input,
and byte guards fail generation if it changes the hint inputs or CommandRegistry.
Selection prose may choose a more or less restrictive recommendation. It cannot
create a Cobra command or flag, change parameter facts, invent an
RPC/interface, alter safety metadata, or bypass command completeness. Examples
must use an executable primary/alias path and flags accepted by the live Cobra
command; never add `--yes` to stored examples.
Every example is always checked against its real `BoundCommand`: exact path,
accepted flags, Cobra required flags/positionals, and the effective
`require_one_of`, `require_together`, and `mutually_exclusive` constraints must
all pass before execution eligibility is considered. A missing required value,
constraint failure, runtime error, or MCP resolution error is a contract bug;
none is a valid reason to skip an example.
Example execution defaults to contract validation only. Runtime execution is
opt-in: an example enters `dry_run` only when its final `ToolSpec` publishes an
explicit reviewed dry-run capability. The test never injects `--yes`, and
`risk`/`confirmation` values do not manufacture preview support. A narrow
runtime precondition that cannot be derived from the typed contract may use an
exact zero-based `example_dispositions` entry with `mode=contract_only`,
`reviewed=true`, one of the schema-enumerated reason codes, and a concrete
non-empty reason. Such a disposition may only narrow an explicit dry-run
capability; it cannot turn an ordinary contract-only example into a skip.
Duplicate, missing, and out-of-range indexes fail validation. Never catch a
dry-run failure and dynamically downgrade it to `contract_only`.
Normal Go tests run the exhaustive contract gate. Run
`make test-schema-agent-examples` to additionally execute the eligible subset
through the real Cobra `--dry-run` path with isolated HOME and blocked proxies.
The test reports stable `total`, `contract`, `dry_run`, `contract_only`,
`reviewed_manual`, and per-reason counts; changing those counts requires a
review of the corresponding typed dry-run capability or manual disposition.
This target is also part of `make policy`.
Treat every tool `use_when` entry as a reviewed positive selection scenario
whose expected result is that tool's canonical path, and every `avoid_when`
entry as a reviewed negative scenario that must not choose that tool. The
deterministic gate derives a typed evaluation fixture from these same fields;
it requires exact tool coverage, a real runnable `BoundCommandRegistry`
primary command, at least one positive and negative assertion per tool, and no
literal contradictory expectations. It does not claim that string matching
proves natural-language understanding.
Semantic selection is an explicit opt-in live-model check. Run the smoke set
(one positive and one negative scenario per product) with
`DWS_AGENT_SELECTION_LIVE=1 ARK_API_KEY=... ARK_BASE_URL=... ARK_MODEL=... go test ./internal/app -run TestManualAgentSelectionArkLive -count=1`.
Add `DWS_AGENT_SELECTION_FULL=1` to evaluate every committed tool scenario, or
set `DWS_AGENT_SELECTION_CASES` to comma-separated fixture case IDs. Normal CI
never calls a model; its blockers remain the reproducible fixture, binding,
example, provenance, and final-delivery facts.
The live evaluator sends only case IDs/scenarios plus one same-product
candidate table; expected/forbidden assertions stay local and must never be
included in the model prompt. Built-in Ark HTTPS bases are allowlisted. A
different HTTPS provider requires its exact base in
`DWS_AGENT_SELECTION_ALLOWED_BASE_URLS`; plaintext HTTP is accepted only for a
loopback test server so API credentials are never sent to an arbitrary clear
text endpoint.
## Safety metadata
Parameter and safety resolution is mostly source-precedence based and
value-neutral: do not choose a winner because one value looks stricter. A
higher-priority reviewed metadata/explicit source may intentionally raise or
lower description, mapping, `effect`, `risk`, `confirmation`, or `idempotency`.
Preserve all candidates and the selected source in provenance, and fail
same-precedence conflicts rather than silently merging them.
`required` is the exception. Cobra `MarkFlagRequired` is a hard floor: the
final Agent projection must keep `required=true` and cannot be lowered by
manual/hint overlays. Overlays may still raise an optional flag to required.
`cli_required` continues to mirror the executable Cobra marker.
For command text, reviewed `ToolSchemaHint` wins first, then command-specific
Cobra Help, then MCP metadata. Generic RPC prose may remain an unselected
provenance candidate (and parameter-level `interface_description`); it must not
overwrite a specialized leaf's title or description.
For every delivered `ToolSpec` and `ParameterSpec` field, the provenance
winner value must exactly equal the delivered value. Checking only source,
count, presence, or hash is not a sufficient final-delivery invariant.
The same resolved `ToolSpec` must drive every projection. The full leaf payload
must equal the corresponding tool in `schema --all` and the full Catalog tool.
Overview/product/group summaries and Catalog summaries must equal
`ToolSpec.ToSummaryPayload()`. An alias lookup may change only the view fields
`cli_path` and `is_alias`; it must not re-resolve or mutate the command
contract.
This build-time rule is distinct from runtime drift handling. If shipped Help
and leaf Schema disagree, pass only flags accepted by Cobra. For conflicting
safety information, do not silently take the less restrictive behavior: use
the safer interpretation or stop and report the contract drift.
Do not infer one safety field from another. In particular, `effect=destructive`
or `risk=high` does not mechanically rewrite `confirmation`; the final
precedence winner for each field is authoritative. When
`confirmation=user_required`, obtain confirmation before adding `--yes`.
Keep CLI confirmation behavior and Schema metadata consistent, and add a
semantic regression test through the final embedded loader/query delivery
path; a generator unit test or JSON count alone is insufficient.
## Current Schema boundaries
- `schema list` remains a progressive overview. `schema --all` is the stable
full-export contract: every final `SchemaIndex` tool must contain its
complete leaf parameters, constraints, and safety semantics, including an empty
`parameters` object for commands without flags. Keep it suitable for the #602
compatibility baseline and fail rather than silently emitting a partial
export.
- `schema --all` is not normal command discovery. Use overview -> product/group
-> leaf for routine Agent work. `--compact` is supported for context-saving
projections, but a compact full export is not a complete compatibility
baseline.
- `dws <path> --help` defines whether Cobra exposes a path and which flags the
executable accepts. A leaf Schema defines Agent selection, parameter mapping
and constraints, and safety/confirmation semantics. A conflict is contract
drift, not permission to guess.
- Schema and Help describe commands; neither returns DingTalk business data.
After discovery, execute the real read/search/list command to obtain data.
For Schema work, the minimum generation entry point is `make generate-schema`.
For CLI path or flag changes, also run
`./scripts/policy/check-command-surface.sh --strict`. Report failures,
environment limits, and unrun checks explicitly in the handoff.
+7 -3
View File
@@ -10,9 +10,13 @@ under the project [Apache License 2.0](./LICENSE).
## Before You Start
1. Read `README.md`.
2. Read the relevant docs under `docs/`.
3. Inspect the code and tests for the area you will change.
4. Decide the smallest safe change that satisfies the request.
2. Normalize the task and select checks with
[`docs/coding-agent-guide.md`](./docs/coding-agent-guide.md).
3. Read the relevant docs under `docs/`. CLI/Schema/Agent metadata work must
also follow
[`docs/schema-contributor-guide.md`](./docs/schema-contributor-guide.md).
4. Inspect the code and tests for the area you will change.
5. Decide the smallest safe change that satisfies the request.
Maintainers and automation authors should also read
`docs/automation.md` for repo-local release and agent workflow
+10 -1
View File
@@ -8,7 +8,7 @@ POLICY_GOTMPDIR ?= $(DWS_POLICY_TMPDIR)/go
POLICY_ENV = DWS_POLICY_TMPDIR="$(DWS_POLICY_TMPDIR)" GOTMPDIR="$(POLICY_GOTMPDIR)"
GO_SOURCE_LIST = git ls-files -z --cached --others --exclude-standard -- '*.go'
.PHONY: all help build rebuild test test-plan lint format-check fmt policy edition-test interface-integrity authoritative-interface-integrity coverage-gate coverage-gate-platform update-interface-baseline reset-interface-baseline schema-compatibility skill-command-integrity cli-smoke mock-mcp-smoke test-schema-agent-examples generate-schema generate-schema-agent-metadata generate-schema-catalog package release release-pre release-stable changelog-pre changelog-stable publish-homebrew-formula setup-hooks
.PHONY: all help build rebuild test test-plan lint format-check fmt policy coding-agent-harness coding-agent-task edition-test interface-integrity authoritative-interface-integrity coverage-gate coverage-gate-platform update-interface-baseline reset-interface-baseline schema-compatibility skill-command-integrity cli-smoke mock-mcp-smoke test-schema-agent-examples generate-schema generate-schema-agent-metadata generate-schema-catalog package release release-pre release-stable changelog-pre changelog-stable publish-homebrew-formula setup-hooks
all: setup-hooks fmt lint build test rebuild
@@ -21,6 +21,8 @@ help:
@printf " make format-check - Check all repository Go source files with gofmt\n"
@printf " make fmt - Format all repository Go source files\n"
@printf " make policy - Check the built dws plus open-source and Schema policies\n"
@printf " make coding-agent-harness - Validate coding-agent task intake, routing, and self-check contracts\n"
@printf " make coding-agent-task TASK=<file> - Validate a filled coding-agent task contract\n"
@printf " make interface-integrity - Check historical commands and help contracts still work\n"
@printf " make authoritative-interface-integrity BASE_REF=<ref> - Check the Git-owned PR merge-base\n"
@printf " make coverage-gate BASE_REF=<ref> - Enforce overall non-regression and 100%% changed-code coverage\n"
@@ -86,6 +88,13 @@ policy:
@$(POLICY_ENV) ./scripts/policy/check-schema-binary.sh
@$(POLICY_ENV) $(MAKE) test-schema-agent-examples
coding-agent-harness:
@./scripts/policy/check-coding-agent-harness.sh
coding-agent-task:
@test -n "$(TASK)" || { printf '%s\n' 'TASK is required, e.g. make coding-agent-task TASK=task.md' >&2; exit 2; }
@./scripts/policy/check-coding-agent-harness.sh -task "$(TASK)"
edition-test:
$(GO) test -v -count=1 ./pkg/editiontest/...
+107
View File
@@ -0,0 +1,107 @@
# Coding Agent Workflow
This is the task intake and self-check contract for coding agents working in
this repository. It is intentionally independent of external Wiki systems:
the checked-out repository is the execution context and evidence source.
## 1. Normalize the task input
Start from the copyable
[`coding-agent-task-template.md`](coding-agent-task-template.md). Keep one
primary outcome per task. Fill unknown fields from the issue, nearby code,
tests, and versioned docs; state any assumption that can affect behavior.
For a saved, filled template, run `make coding-agent-task TASK=path/to/task.md`.
The local checker rejects missing required fields, unsupported task kinds, and
unresolved placeholders before implementation begins.
Do not invent acceptance criteria that expand the requested behavior. Stop for
user input only when the unresolved choice would change externally visible
behavior, compatibility, destructive scope, credentials, or external state.
## 2. Establish the baseline
1. Run `git status --short` and identify pre-existing changes.
2. Read the applicable guides linked from the root `AGENTS.md`.
3. Locate the implementation and its closest tests with `rg`/`rg --files`.
4. Reproduce the bug or capture the current contract before changing it.
5. Pick the smallest owning layer; avoid duplicating policy in a caller when a
shared typed layer already owns it.
Never clean, overwrite, stage, or reformat unrelated user changes. If a
required file is already modified, inspect the overlap and preserve both
intents or stop with the exact conflict.
## 3. Implement from authoritative inputs
- Go behavior belongs in the package that owns the contract, with focused
tests beside it.
- Public CLI paths and flags must match the live Cobra tree and compatibility
policies.
- Schema and Agent-facing changes start from reviewed source inputs; generated
outputs are publication artifacts.
- Documentation describes behavior that exists in the same change.
- Secrets, tokens, local identities, and private endpoints must not enter code,
fixtures, logs, or handoff output.
For generated files, run the repository generator and inspect the resulting
diff. A large or unrelated generated diff is a signal to stop and find the
wrong input or nondeterminism, not something to accept automatically.
## 4. Select validation by change surface
Run the narrow check while iterating, then the applicable admission checks
before handoff. `make help` is the authoritative target list.
| Changed surface | Focused check | Admission checks |
|---|---|---|
| Documentation only | inspect links/examples | `git diff --check` |
| Go implementation | `go test ./path/to/package` | `make format-check`, `make test` |
| CLI paths or flags | focused command/help tests | `make build`, `./scripts/policy/check-command-surface.sh --strict`, `make interface-integrity` |
| Schema registry, hints, or generators | focused generator/app tests | `make generate-schema`, `./scripts/policy/check-generated-drift.sh`, `./scripts/policy/check-schema-catalog.sh`, `make test-schema-agent-examples` |
| Skill command examples | inspect referenced `dws` help | `make skill-command-integrity` |
| CI or test sharding | run the affected script/test | `make test-plan`, `make lint`, and the CI workflow's pinned actionlint command when workflows change |
| Packaging or installers | focused release-script tests | `make package`, `./scripts/release/verify-package-managers.sh` |
| Authentication, transport, or OS-specific code | focused tests, including failure paths | `make test`; run relevant platform checks or disclose the unavailable platform |
`make policy` is the combined policy gate and is appropriate for command,
Schema, generated-asset, or broad cross-cutting changes. Platform credentials
and live services are not prerequisites for ordinary unit tests; never turn a
missing credential into permission to skip deterministic checks.
The guide contract itself is executable. Run `make coding-agent-harness` after
changing `AGENTS.md`, this guide, the Schema contributor guide, or their routed
commands and paths. This remains a local, opt-in check and is not wired into CI.
## Design references
This repository adapts two patterns without copying their product-specific
rules:
- [Lark CLI's contributor guide](https://github.com/larksuite/cli/blob/5efaf65aec59c33899475bb90e6bff1bc3b5b65c/AGENTS.md): one primary goal, machine-consumable errors/output, and validation selected by behavior surface.
- [WeCom CLI's root routing guide](https://github.com/WecomTeam/wecom-cli/blob/9eb7898b959861af879495e211e37431fa908f19/AGENTS.md) and [human helper template](https://github.com/WecomTeam/wecom-cli/blob/9eb7898b959861af879495e211e37431fa908f19/src/helpers/HUMANS.md): a thin root guide, scoped implementation guidance, and a copyable request format.
## 5. Pre-handoff self-check
Confirm every applicable item:
- The diff implements the stated goal and no unrelated cleanup.
- Pre-existing changes are still present and were not attributed to this task.
- New behavior has a regression test; removed behavior has an explicit reason.
- Public command paths, flags, output, exit behavior, and compatibility remain
intentional.
- Destructive or mutating operations retain the required confirmation path.
- Generated files came from their reviewed inputs and generation is clean.
- Docs and examples use commands accepted by current help/Schema.
- Errors preserve actionable context without leaking secrets.
- `git diff --check` passes and the final diff has been read.
- Every reported check is labeled passed, failed, or not run with a reason.
Use this compact handoff shape:
```text
Outcome: what is now true
Files: intentional files changed
Validation: exact commands and results
Limits: unrun checks, environment constraints, follow-ups
```
+35
View File
@@ -0,0 +1,35 @@
# Coding Agent Task Template
Copy this block into an issue or coding-agent request. One task should have one
primary outcome; split unrelated outcomes instead of hiding them in acceptance
criteria.
```text
Task kind: bug | feature | refactor | docs | policy | release
Goal (one primary outcome):
Current behavior and evidence:
Acceptance criteria:
In scope (packages/files/surfaces):
Out of scope:
Compatibility constraints:
Interface impact (commands/flags/output/errors/exit codes/Schema):
Safety or data-mutation constraints:
Expected validation:
Known environment limitations:
```
For a command or remote-interface task, add the smallest concrete invocation
and contract evidence available:
```text
CLI path and example argv:
Current --help or Schema excerpt:
Remote method and request/response shape, if relevant:
Expected stdout/stderr and exit behavior:
Mutation preview/confirmation behavior:
```
Do not paste credentials, tokens, private endpoints, or production business
data. Use redacted fixtures and say which evidence is unavailable. Save the
filled block and run `make coding-agent-task TASK=path/to/task.md` to validate
it before implementation.
+150
View File
@@ -0,0 +1,150 @@
# Schema and Agent Contract Contributor Guide
Read this guide when changing public CLI commands, Schema identity or
parameters, Agent selection/safety metadata, or generated Schema assets.
## Ownership and data flow
The publication graph is one way:
```text
Cobra command tree
+ reviewed CommandRegistry identity/navigation
+ reviewed metadata parameter overlays
-> EffectiveCommandRegistry and executable binding
+ parameter bindings
+ reviewed selection and safety/interface metadata
+ pinned MCP metadata
-> one resolved ToolSpec registry/index
-> generated Agent metadata and embedded Schema Catalog
-> dws schema projections and runtime metadata lookup
```
The owning sources are:
| Concern | Authoritative input |
|---|---|
| Executable paths and accepted flags | Cobra tree built by `app.NewRootCommand()` |
| Stable canonical identity, primary path, aliases, navigation | `internal/cli/schema_command_registry.json` |
| Registry editing contract | `internal/cli/schema_command_registry.schema.json` |
| Safety, interface, runtime gates, parameter overlays | `internal/cli/schema_hints/metadata/<product>.json` |
| Agent selection prose and examples | `internal/cli/schema_hints/selection/<product>.json` |
| Product-to-hint-file routing | `internal/cli/schema_hints/index.json` |
| Flag/property bindings | `internal/cli/schema_parameter_bindings.json` |
| Sanitized interface fallback | `internal/cli/schema_mcp_metadata.json` |
| Exact reviewed omissions from Schema | `internal/cli/schema_command_exclusions.json` |
Generated files under `internal/cli/schema_agent_metadata/` and
`internal/cli/schema_catalog.json` are output only. Runtime loading is a delivery
boundary: it must not create or repair commands, flags, registry entries, or
generation inputs.
## Invariants
1. Every delivered tool resolves to a public runnable Cobra leaf.
2. Every public runnable Cobra leaf resolves to Schema or has one exact,
reviewed exclusion with a non-empty reason. Wildcard/prefix exclusions are
forbidden.
3. The reviewed CommandRegistry is the only stable identity/navigation source.
Native annotations, when present, are consistency assertions and must agree.
4. Metadata overlays may describe or constrain real flags; they cannot create
commands, flags, interfaces, or unknown RPCs.
5. Cobra-required flags are a hard floor. An overlay may make an optional flag
required but cannot make a Cobra-required flag optional.
6. Each tool is resolved once into one typed `ToolSpec`; all Catalog, `schema
--all`, leaf, summary, safety, and runtime projections derive from it.
7. Provenance winner values must equal delivered values. Same-precedence
conflicts fail instead of being merged silently.
8. `confirmation=user_required` requires user confirmation before `--yes`.
Do not infer confirmation mechanically from risk/effect; keep runtime gates
and published metadata consistent.
9. Stored examples use real primary/alias paths and accepted flags, satisfy all
required/constraint rules, contain no shell comments, and never add `--yes`.
10. `schema --all` remains the complete compatibility export. Routine discovery
should use overview, product/group, then leaf queries.
When Help and shipped Schema disagree, treat it as contract drift. Cobra still
defines executable flags; use the safer interpretation for confirmation or
stop rather than guessing.
Parameter and safety resolution is source-precedence based and otherwise
value-neutral: a value must not win merely because it looks stricter. Preserve
all candidates and the selected source, and fail same-precedence conflicts.
Command text resolves from reviewed tool hints, then command-specific Cobra
help, then MCP metadata; generic RPC prose must not replace a specialized
leaf's description. An alias lookup may change only view fields such as
`cli_path` and `is_alias`, never the resolved command contract.
## Editing workflow
1. Confirm the live path and flags in the Cobra tree and current `--help`.
2. Change only the owning reviewed block. Do not copy generated Catalog fields
into inputs or mix selection fields into metadata files.
3. Keep registry edits limited to intentional identity/navigation changes.
4. For selection prose, write decision-oriented routing: when to choose the
command, when a sibling is better, and the result shape. Do not restate help.
5. For parameter overlays, use an exact runnable leaf and real flags; set
`reviewed: true` with a concrete review reason.
6. Regenerate the complete snapshot; publication is deterministic even when
only one product input changed.
7. Inspect authored and generated diffs separately, then run the gates below.
Pinned MCP metadata is a sanitized fallback. When a task requires refreshing
it and a personal session is available, inspect live metadata with `dws auth
status`, `dws cache refresh`, and `dws schema <canonical> -f json`. Never print
or commit tokens. Evidence precedence is Runtime/Cobra, live MCP, pinned MCP,
then Skill prose as evidence only.
## Required checks
```bash
make generate-schema
./scripts/policy/check-runtime-confirmation-truth.sh
./scripts/policy/check-generated-drift.sh
./scripts/policy/check-schema-catalog.sh
./scripts/policy/check-command-surface.sh --strict
make test-schema-agent-examples
```
Also run focused tests for the changed binder, generator, command, or runtime
consumer. Run reverse-completeness tests whenever the Cobra tree changes.
Agent examples are contract-checked by default. Eligible reviewed dry-run
examples are additionally exercised by `make test-schema-agent-examples` with
isolated state; a runtime failure must not be converted into an ad hoc skip.
Live-model selection evaluation is optional and never a normal CI dependency.
An example enters runtime dry-run only when its final typed contract publishes
an explicit reviewed dry-run capability. Risk or confirmation metadata does
not manufacture preview support, and the harness never injects `--yes`. A
narrow precondition that cannot be derived from the contract may use an exact,
reviewed `example_dispositions` entry to narrow dry-run to contract-only; it
must not become a general skip or a fallback applied after execution fails.
Every `use_when` entry is a positive selection fixture and every `avoid_when`
entry is a negative fixture for that tool. The deterministic gate checks
coverage and contradictions; it does not claim to prove natural-language
understanding. When explicitly requested, the optional live-model smoke test
can be run with:
```bash
DWS_AGENT_SELECTION_LIVE=1 \
ARK_API_KEY=... ARK_BASE_URL=... ARK_MODEL=... \
go test ./internal/app -run TestManualAgentSelectionArkLive -count=1
```
Use `DWS_AGENT_SELECTION_FULL=1` for the full fixture or
`DWS_AGENT_SELECTION_CASES=<comma-separated-ids>` for selected cases. A custom
HTTPS provider must be explicitly allowlisted; plaintext is accepted only for
a loopback test server so credentials are not sent over arbitrary clear text.
## Runtime boundaries
- `schema list` is a progressive overview; `schema --all` is the complete,
non-compact compatibility baseline with full parameters, constraints, and
safety semantics.
- `--compact` saves discovery context but is not a full compatibility export.
- `dws <path> --help` decides whether a path and its flags are executable. Leaf
Schema owns Agent selection, mapping, constraints, and safety semantics.
- Help and Schema describe commands; they do not return DingTalk business
data. Execute the real read/list/search command after discovery.
+42
View File
@@ -0,0 +1,42 @@
# Schema Runtime and Publication Agent Guide
This file applies to `internal/cli/`. Read
[`docs/schema-contributor-guide.md`](../../docs/schema-contributor-guide.md)
before editing.
## Owning inputs
| Change | Edit |
|---|---|
| Canonical identity, primary path, aliases, navigation | `schema_command_registry.json` |
| Parameter/property mapping | `schema_parameter_bindings.json` or reviewed metadata overlay |
| Safety, interface, runtime gate | `schema_hints/metadata/<product>.json` |
| Agent selection and examples | `schema_hints/selection/<product>.json` |
| Exact reviewed omission | `schema_command_exclusions.json` |
| Runtime query/projection behavior | Go implementation and focused tests in this package |
The executable Cobra tree outside this package owns whether a command exists
and which flags it accepts. Do not create a command or flag in Schema inputs.
## Generated boundary
- `schema_catalog.json` and `schema_agent_metadata/` are generated outputs.
- Never hand-edit, merge from, or use a previous generated output as an input.
- Change the owning reviewed input or generator, run `make generate-schema`,
and inspect authored and generated diffs separately.
- A broad unrelated generated diff is a failure signal, not acceptable churn.
## Self-check
- Every public runnable Cobra leaf is bound or has one exact reviewed
exclusion; every delivered tool binds back to a runnable leaf.
- Registry identity and native annotations agree; aliases do not mutate the
resolved contract.
- Parameters reference real flags and retain Cobra-required floors.
- Selection examples use executable paths/flags and never include `--yes`.
- Runtime confirmation gates and published safety metadata agree.
- Full, compact, summary, alias, and Catalog projections derive from the same
typed `ToolSpec` and preserve provenance winners.
Run the focused package/generator tests and the complete command list in
`docs/schema-contributor-guide.md`, beginning with `make generate-schema`.
+56
View File
@@ -0,0 +1,56 @@
# Product Command Handler Agent Guide
This file applies to `internal/helpers/`. Read the root `AGENTS.md`,
[`CONTRIBUTING.md`](../../CONTRIBUTING.md), and
[`docs/coding-agent-guide.md`](../../docs/coding-agent-guide.md) first.
## Scope and routing
`internal/helpers` owns product command construction and handler behavior. Find
the owning product file and its closest tests before editing.
| Concern | Owning surface |
|---|---|
| Root/static command wiring or plugin loading | `internal/app` |
| Product command flags and handler behavior | `internal/helpers` |
| Shared Cobra construction | `internal/cobracmd` |
| Invocation and transport | `internal/executor`, `internal/transport` |
| Structured failures and recovery hints | `internal/errors`, `internal/recovery` |
| Output encoding and projections | `internal/output` |
| Confirmation and dry-run guards | `internal/safety` and command runtime gates |
| Agent identity/Schema/parameters | `internal/cli` and [`docs/schema-contributor-guide.md`](../../docs/schema-contributor-guide.md) |
Do not modify `internal/app` or shared layers merely to register an ordinary
product leaf when the existing helper construction already owns it. Cross the
package boundary only when the task changes that shared contract.
## Command contract
- Confirm the current path and flags with `dws <path> --help` and the Cobra
construction before changing behavior.
- Keep stdout machine-readable business data. Send progress, warnings, and
diagnostics through the established stderr/logging paths.
- Preserve structured error category, stable exit behavior, cause, trace ID,
and an actionable recovery hint. Do not replace a typed lower-layer failure
with an unclassified message.
- Treat paths, JSON payloads, filenames, and remote content as untrusted input;
use existing validation and sanitization helpers.
- Mutating or destructive commands must keep their preview/confirmation
behavior aligned with runtime safety and published Schema metadata.
- If flags, identity, parameters, selection, or safety metadata change, follow
the scoped `internal/cli/AGENTS.md` and regenerate from reviewed inputs.
## Verification matrix
| Change | Required focused evidence |
|---|---|
| Handler bug or behavior | package regression test including the failure path |
| Flags or request mapping | Cobra/help test plus dry-run request assertion when the command supports preview |
| Output or error behavior | structured payload, stderr/stdout, and exit-code assertions |
| Mutating behavior | preview/confirmation test; live execution only with explicit authorization and disposable data |
| Shared helper refactor | affected product tests plus `go test ./internal/helpers/...` |
| Public command surface | Schema/command gates from `internal/cli/AGENTS.md` |
Before handoff, run the narrow package tests, `make format-check`, and the
admission checks selected by `docs/coding-agent-guide.md`. Do not claim a live
round-trip when only dry-run or mocked transport was exercised.
+7
View File
@@ -0,0 +1,7 @@
#!/bin/sh
set -eu
ROOT="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
cd "$ROOT"
exec go run ./scripts/policy/coding-agent-harness -root "$ROOT" "$@"
+403
View File
@@ -0,0 +1,403 @@
package main
import (
"bufio"
"errors"
"flag"
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
)
var guideLineLimits = map[string]int{
"AGENTS.md": 80,
"internal/cli/AGENTS.md": 80,
"internal/helpers/AGENTS.md": 100,
}
var markdownLinkPattern = regexp.MustCompile(`\[[^]]+\]\(([^)]+)\)`)
type fileContract struct {
path string
required []string
}
type taskField struct {
label string
allowNone bool
allowedValue map[string]bool
}
var taskFields = []taskField{
{label: "Task kind", allowedValue: map[string]bool{"bug": true, "feature": true, "refactor": true, "docs": true, "policy": true, "release": true}},
{label: "Goal (one primary outcome)"},
{label: "Current behavior and evidence"},
{label: "Acceptance criteria"},
{label: "In scope (packages/files/surfaces)"},
{label: "Out of scope", allowNone: true},
{label: "Compatibility constraints", allowNone: true},
{label: "Interface impact (commands/flags/output/errors/exit codes/Schema)", allowNone: true},
{label: "Safety or data-mutation constraints", allowNone: true},
{label: "Expected validation"},
{label: "Known environment limitations", allowNone: true},
}
var guideContracts = []fileContract{
{
path: "AGENTS.md",
required: []string{
"docs/coding-agent-guide.md",
"docs/coding-agent-task-template.md",
"docs/schema-contributor-guide.md",
"internal/helpers/AGENTS.md",
"docs/automation.md",
"docs/agent-code.md",
"Do not depend on generated Wiki or CodeWiki content.",
},
},
{
path: "CONTRIBUTING.md",
required: []string{
"docs/coding-agent-guide.md",
"docs/schema-contributor-guide.md",
},
},
{
path: "docs/coding-agent-guide.md",
required: []string{
"## 1. Normalize the task input",
"coding-agent-task-template.md",
"make coding-agent-task TASK=",
"primary outcome per task",
"github.com/larksuite/cli/blob/",
"github.com/WecomTeam/wecom-cli/blob/",
"## 2. Establish the baseline",
"## 3. Implement from authoritative inputs",
"## 4. Select validation by change surface",
"Documentation only",
"Go implementation",
"CLI paths or flags",
"Schema registry, hints, or generators",
"CI or test sharding",
"Packaging or installers",
"Authentication, transport, or OS-specific code",
"## 5. Pre-handoff self-check",
"Outcome: what is now true",
"Validation: exact commands and results",
},
},
{
path: "docs/coding-agent-task-template.md",
required: []string{
"Goal (one primary outcome):",
"Current behavior and evidence:",
"Acceptance criteria:",
"In scope (packages/files/surfaces):",
"Out of scope:",
"Compatibility constraints:",
"Interface impact (commands/flags/output/errors/exit codes/Schema):",
"Safety or data-mutation constraints:",
"Expected validation:",
"Known environment limitations:",
"Expected stdout/stderr and exit behavior:",
"Mutation preview/confirmation behavior:",
},
},
{
path: "docs/schema-contributor-guide.md",
required: []string{
"internal/cli/schema_command_registry.json",
"internal/cli/schema_command_registry.schema.json",
"internal/cli/schema_hints/metadata/<product>.json",
"internal/cli/schema_hints/selection/<product>.json",
"internal/cli/schema_parameter_bindings.json",
"internal/cli/schema_mcp_metadata.json",
"internal/cli/schema_command_exclusions.json",
"internal/cli/schema_catalog.json",
"make generate-schema",
"make test-schema-agent-examples",
},
},
{
path: "internal/helpers/AGENTS.md",
required: []string{
"../../CONTRIBUTING.md",
"../../docs/coding-agent-guide.md",
"../../docs/schema-contributor-guide.md",
"Keep stdout machine-readable business data.",
"structured error category",
"preview/confirmation",
"Do not claim a live",
},
},
{
path: "internal/cli/AGENTS.md",
required: []string{
"../../docs/schema-contributor-guide.md",
"schema_command_registry.json",
"schema_parameter_bindings.json",
"schema_hints/metadata/<product>.json",
"schema_hints/selection/<product>.json",
"schema_command_exclusions.json",
"schema_catalog.json",
"make generate-schema",
},
},
}
var requiredPaths = []string{
"docs/automation.md",
"docs/agent-code.md",
"scripts/policy/check-runtime-confirmation-truth.sh",
"scripts/policy/check-coding-agent-harness.sh",
"scripts/policy/check-generated-drift.sh",
"scripts/policy/check-schema-catalog.sh",
"scripts/policy/check-command-surface.sh",
"scripts/release/verify-package-managers.sh",
}
var requiredMakeTargets = []string{
"build",
"coding-agent-harness",
"coding-agent-task",
"format-check",
"test",
"policy",
"interface-integrity",
"skill-command-integrity",
"test-schema-agent-examples",
"generate-schema",
"package",
}
func main() {
root := flag.String("root", ".", "repository root")
task := flag.String("task", "", "optional filled coding-agent task file to validate")
flag.Parse()
if flag.NArg() != 0 {
fmt.Fprintln(os.Stderr, "coding agent harness: unexpected positional arguments")
os.Exit(2)
}
problems := validate(*root)
if strings.TrimSpace(*task) != "" {
taskPath := *task
if !filepath.IsAbs(taskPath) {
taskPath = filepath.Join(*root, taskPath)
}
problems = append(problems, validateTask(taskPath)...)
sort.Slice(problems, func(i, j int) bool { return problems[i].Error() < problems[j].Error() })
}
if len(problems) != 0 {
fmt.Fprintln(os.Stderr, "coding agent harness: failed")
for _, problem := range problems {
fmt.Fprintf(os.Stderr, "- %v\n", problem)
}
os.Exit(1)
}
fmt.Println("coding agent harness: ok")
}
func validateTask(path string) []error {
content, err := os.ReadFile(path)
if err != nil {
return []error{fmt.Errorf("read task contract %s: %w", path, err)}
}
values := make(map[string][]string)
occurrences := make(map[string]int)
current := ""
scanner := bufio.NewScanner(strings.NewReader(string(content)))
for scanner.Scan() {
line := strings.TrimSpace(scanner.Text())
matched := false
for _, field := range taskFields {
prefix := field.label + ":"
if strings.HasPrefix(line, prefix) {
current = field.label
occurrences[current]++
inline := strings.TrimSpace(strings.TrimPrefix(line, prefix))
if inline != "" {
values[current] = append(values[current], inline)
}
matched = true
break
}
}
if matched || current == "" || line == "" || strings.HasPrefix(line, "```") {
continue
}
values[current] = append(values[current], line)
}
if err := scanner.Err(); err != nil {
return []error{fmt.Errorf("scan task contract %s: %w", path, err)}
}
var problems []error
for _, field := range taskFields {
if occurrences[field.label] > 1 {
problems = append(problems, fmt.Errorf("task contract field %q appears %d times; each field must be unique", field.label, occurrences[field.label]))
}
value := strings.TrimSpace(strings.Join(values[field.label], "\n"))
if value == "" {
problems = append(problems, fmt.Errorf("task contract is missing a value for %q", field.label))
continue
}
normalized := strings.ToLower(value)
if len(field.allowedValue) != 0 && !field.allowedValue[normalized] {
problems = append(problems, fmt.Errorf("task contract field %q has unsupported value %q", field.label, value))
}
if isPlaceholderTaskValue(normalized) {
problems = append(problems, fmt.Errorf("task contract field %q still contains placeholder value %q", field.label, value))
}
if !field.allowNone && (normalized == "none" || normalized == "n/a" || normalized == "not applicable") {
problems = append(problems, fmt.Errorf("task contract field %q requires concrete evidence", field.label))
}
}
sort.Slice(problems, func(i, j int) bool { return problems[i].Error() < problems[j].Error() })
return problems
}
func isPlaceholderTaskValue(value string) bool {
for _, line := range strings.Split(value, "\n") {
trimmed := strings.TrimSpace(line)
trimmed = strings.TrimLeft(trimmed, "-*0123456789.) ")
switch trimmed {
case "todo", "tbd", "unknown", "fill me", "to be decided":
return true
}
if strings.HasPrefix(trimmed, "todo:") || strings.HasPrefix(trimmed, "tbd:") || strings.Contains(trimmed, "<fill") || strings.Contains(trimmed, "<todo") {
return true
}
}
return false
}
func validate(root string) []error {
absRoot, err := filepath.Abs(root)
if err != nil {
return []error{fmt.Errorf("resolve root: %w", err)}
}
var problems []error
for _, contract := range guideContracts {
content, readErr := os.ReadFile(filepath.Join(absRoot, filepath.FromSlash(contract.path)))
if readErr != nil {
problems = append(problems, fmt.Errorf("read %s: %w", contract.path, readErr))
continue
}
text := string(content)
for _, required := range contract.required {
if !strings.Contains(text, required) {
problems = append(problems, fmt.Errorf("%s is missing required contract text %q", contract.path, required))
}
}
problems = append(problems, validateLocalLinks(absRoot, contract.path, text)...)
}
for path, limit := range guideLineLimits {
if lineCount, countErr := countLines(filepath.Join(absRoot, filepath.FromSlash(path))); countErr != nil {
problems = append(problems, fmt.Errorf("count %s lines: %w", path, countErr))
} else if lineCount > limit {
problems = append(problems, fmt.Errorf("%s has %d lines; scoped guide limit is %d", path, lineCount, limit))
}
}
for _, path := range requiredPaths {
info, statErr := os.Stat(filepath.Join(absRoot, filepath.FromSlash(path)))
if statErr != nil {
problems = append(problems, fmt.Errorf("required referenced path %s: %w", path, statErr))
} else if strings.HasSuffix(path, ".sh") && info.Mode().Perm()&0o111 == 0 {
problems = append(problems, fmt.Errorf("required referenced script %s is not executable", path))
}
}
makeTargets, makeErr := loadMakeTargets(filepath.Join(absRoot, "Makefile"))
if makeErr != nil {
problems = append(problems, makeErr)
} else {
for _, target := range requiredMakeTargets {
if !makeTargets[target] {
problems = append(problems, fmt.Errorf("makefile is missing referenced target %q", target))
}
}
}
sort.Slice(problems, func(i, j int) bool { return problems[i].Error() < problems[j].Error() })
return problems
}
func validateLocalLinks(root, document, content string) []error {
var problems []error
documentDir := filepath.Join(root, filepath.Dir(filepath.FromSlash(document)))
for _, match := range markdownLinkPattern.FindAllStringSubmatch(content, -1) {
target := strings.TrimSpace(match[1])
if target == "" || strings.HasPrefix(target, "#") || strings.HasPrefix(target, "http://") || strings.HasPrefix(target, "https://") || strings.HasPrefix(target, "mailto:") {
continue
}
if cut, _, ok := strings.Cut(target, "#"); ok {
target = cut
}
if target == "" {
continue
}
if filepath.IsAbs(target) {
problems = append(problems, fmt.Errorf("%s contains absolute local link %q", document, match[1]))
continue
}
resolved := filepath.Clean(filepath.Join(documentDir, filepath.FromSlash(target)))
rel, err := filepath.Rel(root, resolved)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
problems = append(problems, fmt.Errorf("%s link %q escapes repository root", document, match[1]))
continue
}
if _, err := os.Stat(resolved); err != nil {
problems = append(problems, fmt.Errorf("%s has broken local link %q: %w", document, match[1], err))
}
}
return problems
}
func countLines(path string) (int, error) {
file, err := os.Open(path)
if err != nil {
return 0, err
}
defer file.Close()
scanner := bufio.NewScanner(file)
count := 0
for scanner.Scan() {
count++
}
return count, scanner.Err()
}
func loadMakeTargets(path string) (map[string]bool, error) {
file, err := os.Open(path)
if err != nil {
return nil, fmt.Errorf("read Makefile targets: %w", err)
}
defer file.Close()
targets := make(map[string]bool)
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
if line == "" || line[0] == '\t' || strings.HasPrefix(strings.TrimSpace(line), "#") {
continue
}
name, _, ok := strings.Cut(line, ":")
if !ok || strings.ContainsAny(name, "= ") || strings.HasPrefix(name, ".") {
continue
}
targets[name] = true
}
if err := scanner.Err(); err != nil {
return nil, errors.New("scan Makefile targets: " + err.Error())
}
return targets, nil
}
@@ -0,0 +1,250 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
func TestRepositoryCodingAgentHarness(t *testing.T) {
root, err := filepath.Abs(filepath.Join("..", "..", ".."))
if err != nil {
t.Fatalf("resolve repository root: %v", err)
}
if problems := validate(root); len(problems) != 0 {
t.Fatalf("repository harness problems:\n%s", formatProblems(problems))
}
}
func TestCodingAgentHarnessFixturePasses(t *testing.T) {
root := newHarnessFixture(t)
if problems := validate(root); len(problems) != 0 {
t.Fatalf("valid fixture problems:\n%s", formatProblems(problems))
}
}
func TestCodingAgentTaskContractPasses(t *testing.T) {
path := filepath.Join(t.TempDir(), "task.md")
writeTaskFixture(t, path, validTaskContract)
if problems := validateTask(path); len(problems) != 0 {
t.Fatalf("valid task problems:\n%s", formatProblems(problems))
}
}
func TestCodingAgentTaskContractFailsClosed(t *testing.T) {
tests := []struct {
name string
content string
wantProblem string
}{
{
name: "missing goal",
content: strings.Replace(validTaskContract, "Goal (one primary outcome): Fix retry handling\n", "", 1),
wantProblem: `missing a value for "Goal (one primary outcome)"`,
},
{
name: "blank acceptance criteria",
content: strings.Replace(validTaskContract, "Acceptance criteria:\n- The retry succeeds once and then stops.\n", "Acceptance criteria:\n", 1),
wantProblem: `missing a value for "Acceptance criteria"`,
},
{
name: "unsupported task kind",
content: strings.Replace(validTaskContract, "Task kind: bug", "Task kind: experiment", 1),
wantProblem: `unsupported value "experiment"`,
},
{
name: "placeholder acceptance bullet",
content: strings.Replace(validTaskContract, "- The retry succeeds once and then stops.", "- TBD: decide expected retry result", 1),
wantProblem: "still contains placeholder value",
},
{
name: "duplicate goal",
content: strings.Replace(validTaskContract, "Goal (one primary outcome): Fix retry handling", "Goal (one primary outcome): Fix retry handling\nGoal (one primary outcome): Refactor transport", 1),
wantProblem: `field "Goal (one primary outcome)" appears 2 times`,
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "task.md")
writeTaskFixture(t, path, test.content)
problems := formatProblems(validateTask(path))
if !strings.Contains(problems, test.wantProblem) {
t.Fatalf("problems:\n%s\nwant substring %q", problems, test.wantProblem)
}
})
}
}
func TestCodingAgentHarnessFailsClosed(t *testing.T) {
tests := []struct {
name string
mutate func(t *testing.T, root string)
wantProblem string
}{
{
name: "root guide is no longer thin",
mutate: func(t *testing.T, root string) {
path := filepath.Join(root, "AGENTS.md")
file, err := os.OpenFile(path, os.O_APPEND|os.O_WRONLY, 0)
if err != nil {
t.Fatal(err)
}
defer file.Close()
for i := 0; i < guideLineLimits["AGENTS.md"]; i++ {
if _, err := file.WriteString("extra line\n"); err != nil {
t.Fatal(err)
}
}
},
wantProblem: "scoped guide limit",
},
{
name: "task input field is removed",
mutate: func(t *testing.T, root string) {
replaceFixtureText(t, root, "docs/coding-agent-task-template.md", "Acceptance criteria:", "")
},
wantProblem: `docs/coding-agent-task-template.md is missing required contract text "Acceptance criteria:"`,
},
{
name: "route link is broken",
mutate: func(t *testing.T, root string) {
replaceFixtureText(t, root, "AGENTS.md", "[automation](docs/automation.md)", "[automation](docs/missing.md)")
},
wantProblem: "broken local link",
},
{
name: "referenced script is absent",
mutate: func(t *testing.T, root string) {
if err := os.Remove(filepath.Join(root, "scripts/policy/check-schema-catalog.sh")); err != nil {
t.Fatal(err)
}
},
wantProblem: "required referenced path scripts/policy/check-schema-catalog.sh",
},
{
name: "make target is absent",
mutate: func(t *testing.T, root string) {
replaceFixtureText(t, root, "Makefile", "package:\n", "removed-package:\n")
},
wantProblem: `makefile is missing referenced target "package"`,
},
{
name: "harness script is not executable",
mutate: func(t *testing.T, root string) {
path := filepath.Join(root, "scripts/policy/check-coding-agent-harness.sh")
if err := os.Chmod(path, 0o644); err != nil {
t.Fatal(err)
}
},
wantProblem: "required referenced script scripts/policy/check-coding-agent-harness.sh is not executable",
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
root := newHarnessFixture(t)
test.mutate(t, root)
problems := formatProblems(validate(root))
if !strings.Contains(problems, test.wantProblem) {
t.Fatalf("problems:\n%s\nwant substring %q", problems, test.wantProblem)
}
})
}
}
func newHarnessFixture(t *testing.T) string {
t.Helper()
root := t.TempDir()
for _, contract := range guideContracts {
var content strings.Builder
content.WriteString("# Fixture\n\n")
for _, required := range contract.required {
content.WriteString(required)
content.WriteByte('\n')
}
if contract.path == "AGENTS.md" {
content.WriteString("[coding](docs/coding-agent-guide.md)\n")
content.WriteString("[schema](docs/schema-contributor-guide.md)\n")
content.WriteString("[automation](docs/automation.md)\n")
content.WriteString("[agent code](docs/agent-code.md)\n")
}
writeFixtureFile(t, root, contract.path, content.String())
}
for _, path := range requiredPaths {
writeFixtureFileMode(t, root, path, "fixture\n", 0o755)
}
var makefile strings.Builder
for _, target := range requiredMakeTargets {
makefile.WriteString(target)
makefile.WriteString(":\n\t@true\n")
}
writeFixtureFile(t, root, "Makefile", makefile.String())
return root
}
func writeFixtureFile(t *testing.T, root, path, content string) {
t.Helper()
writeFixtureFileMode(t, root, path, content, 0o644)
}
func writeFixtureFileMode(t *testing.T, root, path, content string, mode os.FileMode) {
t.Helper()
full := filepath.Join(root, filepath.FromSlash(path))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatalf("create fixture directory: %v", err)
}
if err := os.WriteFile(full, []byte(content), mode); err != nil {
t.Fatalf("write fixture %s: %v", path, err)
}
}
func replaceFixtureText(t *testing.T, root, path, old, replacement string) {
t.Helper()
full := filepath.Join(root, filepath.FromSlash(path))
content, err := os.ReadFile(full)
if err != nil {
t.Fatalf("read fixture %s: %v", path, err)
}
updated := strings.Replace(string(content), old, replacement, 1)
if updated == string(content) {
t.Fatalf("fixture %s does not contain %q", path, old)
}
if err := os.WriteFile(full, []byte(updated), 0o644); err != nil {
t.Fatalf("update fixture %s: %v", path, err)
}
}
func formatProblems(problems []error) string {
var lines []string
for _, problem := range problems {
lines = append(lines, problem.Error())
}
return strings.Join(lines, "\n")
}
func writeTaskFixture(t *testing.T, path, content string) {
t.Helper()
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatalf("write task fixture: %v", err)
}
}
const validTaskContract = `# Task
Task kind: bug
Goal (one primary outcome): Fix retry handling
Current behavior and evidence: The focused test reproduces two retries.
Acceptance criteria:
- The retry succeeds once and then stops.
In scope (packages/files/surfaces): internal/transport
Out of scope: Authentication behavior
Compatibility constraints: Preserve existing flags and output
Interface impact (commands/flags/output/errors/exit codes/Schema): None
Safety or data-mutation constraints: No external writes
Expected validation: Focused unit test and make test
Known environment limitations: Live service unavailable
`
@@ -0,0 +1,15 @@
# Retry Handling Task
Task kind: bug
Goal (one primary outcome): Fix transient retry handling
Current behavior and evidence: A focused transport test reproduces two retries after a successful response.
Acceptance criteria:
- The retry loop stops after the first successful response.
- The existing output and exit behavior remain unchanged.
In scope (packages/files/surfaces): internal/transport retry implementation and tests
Out of scope: Authentication and endpoint discovery
Compatibility constraints: Preserve existing flags, JSON output, and exit codes
Interface impact (commands/flags/output/errors/exit codes/Schema): None
Safety or data-mutation constraints: No live service calls or external writes
Expected validation: Focused transport tests, race test, formatting, and full Go test suite
Known environment limitations: Live DingTalk service is intentionally not used