Compare commits

...
Author SHA1 Message Date
玉澜 c7521c1ca3 fix(cli): route Chinese product-name queries to the named product
Natural-language queries that name a product in Chinese (e.g. 「帮我搜一下
「评测项目管理」钉钉AI表格」) lost the named product's candidates to a
sibling product whose summaries repeat a shared bigram (92 sheet tools
repeat 「表格」), crowding aitable out of the Top-K cut entirely.

Two-part fix in the lexical layer, deterministic and no new dependencies:

- Fold the product display name into each tool's identity field at index
  build (runtime assembly carries the display name in Description).
- Query-side named-product guarantee: when the query contains a product's
  display-name core (NFKC-folded, generic suffixes like 「操作」「管理」
  trimmed, longest core wins on overlap), fetch that product's best
  candidate over its own eligible subset and merge it just above the
  global top hit. A post-hoc multiplier alone cannot rescue candidates
  truncated inside Retrieve's CandidateLimit cut.

Eval evidence (core120 first-turn coverage): 101→103/120; all four
measured 「钉钉AI表格」 misses fixed, zero regressions across the battery.
2026-08-19 14:31:48 +08:00
玉澜 aa0bc0c2d4 Merge remote-tracking branch 'upstream/main' into pr984 2026-08-17 21:55:45 +08:00
dxb 9e3a5d6fbd Merge pull request #1029 from DingTalk-Real-AI/fix/chat-sender-identity-contract
fix(chat): preserve unverified sender identity semantics
2026-08-17 19:10:30 +08:00
玉澜 f2a3dd2344 fix(cli): hint tokenized schema path failures at the path-form boundary
- the unknown runtime schema path error now states that canonical paths
  contain dots and must be a single argument while space-separated tokens
  resolve as a CLI path; help text and docs only reach agents that consult
  them, the error is the one surface guaranteed to be seen at failure time
- the app trust diagnostic comment is shortened to the durable constraint
  (do not pin absolute Catalog-size counts) instead of narrating the change
- NewDeliveryToolSearchEngine's doc comment notes the memo serves long-lived
  hosts; the shipped schema search command rebuilds per invocation through
  NewDeliveryToolSearchEngineWithConfig
2026-08-17 19:10:18 +08:00
栩朝 fc05976d33 fix(chat): align chat message selection intent 2026-08-17 17:30:12 +08:00
栩朝 021da02474 fix(chat): preserve unverified sender identity semantics 2026-08-17 17:30:12 +08:00
github-actions[bot] a5b9e5a13f Merge pull request #928 from Anonymity-0/feat/bot-group-reply
feat(chat): support bot group message replies
2026-08-17 17:25:23 +08:00
玉澜 b8c9ff2619 test(app): drop stale Catalog-size sentinels from the trust diagnostic
The diagnostic test pinned absolute counts from one Catalog generation:
1123 intent cases, Recall@5 == 990/1123, 1098 tools and the 402/721
language slices. Merging main grew the Catalog to 1121 tools, so the
population became 1147/407/740 and the test failed as a size drift
rather than as a trust regression.

- the absolute counts are removed; the observed intent, language-slice
  and context values are logged for CI forensics and the headline numbers
  stay in the generated comparison JSON
- identity coverage is still enforced exhaustively, but the expected
  count is derived from report.Context.ToolCount instead of a literal
- the trust contract that does not move with Catalog size stays a hard
  gate: every identity pass rate == 1, alias cases non-zero, all four
  integrity counters == 0, zero-result rate == 0, non-empty language
  slices and ReductionVsFullSchema >= 0.99
2026-08-17 17:21:28 +08:00
玉澜 225a073d89 fix(cli): close tool search self-review findings
- SearchSubqueries returned a nil candidate slice on the
  decomposed_exact_filtered refusal, so that one strategy encoded
  "candidates":null while every other path encoded []; the refusal now
  carries an empty slice and a marshal assertion pins the wire shape
- the memoized delivery engine was never invalidated: it indexes one
  Catalog generation and caches its source/surface hashes, but
  RegisterSchemaSourceRoot -> resetDeliverySchemaCatalogState left the
  memo installed, so a re-registered root could be served stale hashes.
  Catalog delivery reset now clears it, with a ForTest wrapper and a
  regression test for the rebuild
- dangerous filter Unicode is rejected before the effect vocabulary and
  Catalog product checks, so a control/Bidi character returns
  dangerous_filter_unicode instead of being echoed back through
  invalid_effect; the filter list is an ordered slice, not a map, so the
  reported field name is deterministic
- dws schema help and RFC 5.8 record the path-form boundary: multi-token
  args resolve as a CLI path, a canonical path must stay one argument
- --explain and ToolSearchScoreBreakdown declare that the reported score
  is the raw value while ordering uses the quantized key
- --limit/--candidate-limit help state that 0 means "use the default",
  which was already the behavior but read as an asymmetry against the
  rejected negative values
- bootstrap seeds become named constants and are published in both
  evaluation reports (bootstrap_samples/bootstrap_seed/
  recovery_bootstrap_seed) so an interval is reproducible without
  reading the source
- dingtalk-shared documents the cross-product summary --dry-run flag and
  that its command list is a plan, not business data
2026-08-17 17:21:10 +08:00
玉澜 45a52dee85 docs: consolidate the six tool search documents into one file
The RFC, source-implementation research, GitHub architecture research,
ranking evaluation report, 2026-08-14 optimization plan and ten-round
review record were six separate files that cross-linked each other for
the same decision chain. They are merged into
docs/tool-search-progressive-discovery.md as Part 1..6, ordered
decision -> research -> evidence -> plan -> review record.

- cross-file links become in-document "本文档 Part N" pointers; the five
  remaining relative links (source and evaluator paths) still resolve
- section content is unchanged apart from one addition: 5.8 Inspect now
  records the path-form boundary (multi-token args resolve as a CLI path;
  a canonical path must be passed as a single argument), matching the
  dws schema help text
2026-08-17 17:20:41 +08:00
Anonymity-0 9f8c525008 Merge branch 'main' into feat/bot-group-reply 2026-08-17 16:59:05 +08:00
github-actions[bot] 207d4dd7e5 Merge pull request #980 from cywan1998/feat/calendar-event-share-info
feat(calendar): add event share-info command
2026-08-17 08:57:50 +00:00
fengbai 9a6b7d4d41 Merge branch 'main' into feat/calendar-event-share-info 2026-08-17 16:41:08 +08:00
前津 5947016cc1 feat(chat): support bot group message replies 2026-08-17 16:09:08 +08:00
github-actions[bot] 386426bb92 Merge pull request #1012 from DingTalk-Real-AI/dws_0814_1723
fix(skill): update doc and drive descriptions for clearer routing
2026-08-17 07:45:22 +00:00
john 3cea671a54 Merge branch 'main' into dws_0814_1723 2026-08-17 15:27:17 +08:00
john 235e881ebe Merge branch 'main' into test/tool-search-ranking-eval 2026-08-17 15:21:07 +08:00
github-actions[bot] f06ea4d9e2 Merge pull request #960 from DingTalk-Real-AI/codex/doc-reread-audit
fix(doc): harden mutation readback verification
2026-08-17 07:06:53 +00:00
Dennis a82d945f54 fix(doc): reject explicit revert failure states 2026-08-17 14:48:13 +08:00
Dennis 6846326445 fix(doc): reject revert request echo evidence 2026-08-17 14:48:11 +08:00
Dennis 54c2054a5c fix(doc): ignore generated JSONML defaults 2026-08-17 14:48:09 +08:00
Dennis e5bf332b05 fix(doc): address readback review findings 2026-08-17 14:48:06 +08:00
Dennis 9ed55978d9 fix(doc): cancel readback retry waits 2026-08-17 14:48:04 +08:00
Dennis 7ffbbc4a51 test(doc): cover stable pagination identities 2026-08-17 14:48:02 +08:00
Dennis 2db73a8185 fix(doc): distinguish identical pagination pages 2026-08-17 14:48:00 +08:00
Dennis a62332be93 fix(doc): trust only explicit inserted block IDs 2026-08-17 14:47:58 +08:00
Dennis 1083093cbc test(doc): complete readback coverage evidence 2026-08-17 14:47:56 +08:00
Dennis c6ebe307cd fix(doc): verify inline media from jsonml readback 2026-08-17 14:47:54 +08:00
Dennis 3ee66d4373 fix(doc): harden mutation readback verification 2026-08-17 14:47:51 +08:00
github-actions[bot] a0be395ccc Merge pull request #1006 from DingTalk-Real-AI/codex/fix-aitable-pagination-minutes-unshare
fix(shortcut): harden Aitable pagination and Minutes unshare
2026-08-17 06:37:16 +00:00
ruigong 93dbd768f7 fix(skill): add explicit recent-edited route to drive SOP-1 2026-08-17 14:18:03 +08:00
Dennis c1a549cd64 fix: close delete readback continuations 2026-08-17 14:13:51 +08:00
Dennis 5a414999ef fix: validate record query previews 2026-08-17 14:13:49 +08:00
Dennis 7aa8240629 fix: preserve record query preview contract 2026-08-17 14:13:47 +08:00
Dennis 37b9a1dc31 fix: bound exact aitable record queries 2026-08-17 14:13:45 +08:00
Dennis 2ab8748c4d test: use native minutes path separators 2026-08-17 14:13:43 +08:00
Dennis f041275811 fix: make minutes polling portable 2026-08-17 14:13:41 +08:00
Dennis f486105836 fix: bound empty aitable pagination 2026-08-17 14:13:38 +08:00
Dennis e14de2b4c2 test: close shortcut fix review gates 2026-08-17 14:13:36 +08:00
Dennis fe2f3ca92f fix: harden aitable pagination and minutes unshare 2026-08-17 14:13:33 +08:00
github-actions[bot] 8e4519cacd Merge pull request #1014 from FloralTide/codex/fix-windows-event-bus
fix(event): support Windows bus lifecycle
2026-08-17 14:12:46 +08:00
炳昱 16abb481e8 Merge remote-tracking branch 'upstream/main' into codex/fix-windows-event-bus 2026-08-17 13:00:07 +08:00
炳昱 7ad82bbf0a fix(event): accept bus exit at stop timeout boundary 2026-08-17 13:00:07 +08:00
chichuan 104eb715c4 Merge pull request #989 from maoqxxmm/codex/sheet-dropdown-source-range
feat(sheet): support SourceRange dropdowns and read completion
2026-08-17 12:19:00 +08:00
chichuan 97ea887ea5 Merge branch 'main' into codex/sheet-dropdown-source-range 2026-08-17 11:49:42 +08:00
RuiGong01 03838a3430 Merge branch 'main' into dws_0814_1723 2026-08-17 11:39:56 +08:00
github-actions[bot] bfeb9f6af0 chore: update beta formula for v1.0.59-beta.2 [skip ci] 2026-08-17 03:35:41 +00:00
毛球 e26f278112 Merge branch 'main' into codex/sheet-dropdown-source-range 2026-08-17 11:17:45 +08:00
RuiGong01 0d34150333 Merge branch 'main' into dws_0814_1723 2026-08-17 11:16:56 +08:00
chichuan e6b5938bd8 Merge pull request #1025 from DingTalk-Real-AI/codex/changelog-v1.0.59-beta.2
docs: seal changelog for v1.0.59-beta.2
2026-08-17 11:03:27 +08:00
chichuan 4f95373420 docs: seal changelog for v1.0.59-beta.2 2026-08-17 10:59:11 +08:00
炳昱 afb90009f6 Merge remote-tracking branch 'upstream/main' into codex/fix-windows-event-bus 2026-08-17 10:49:25 +08:00
RuiGong01 9e8b58cbb6 Merge branch 'main' into dws_0814_1723 2026-08-17 10:47:33 +08:00
github-actions[bot] 6411d26a95 Merge pull request #1023 from DingTalk-Real-AI/fix/app-partition-parallel-jobs
fix(ci): parallelize app test partitions and drop race from the schema partition
2026-08-17 02:45:57 +00:00
毛球 31117d1b89 Merge branch 'main' into codex/sheet-dropdown-source-range 2026-08-17 10:43:01 +08:00
炳昱 e3553fe7a5 test(event): cover bus ownership validation failures 2026-08-17 10:40:21 +08:00
RuiGong01 fe724e96e8 Merge branch 'main' into dws_0814_1723 2026-08-17 10:39:02 +08:00
炳昱 067aff179f Merge remote-tracking branch 'upstream/main' into codex/fix-windows-event-bus 2026-08-17 10:32:08 +08:00
炳昱 353454abb2 fix(event): verify bus owner before fallback stop 2026-08-17 10:32:04 +08:00
chichuan 96b9cbce02 Merge branch 'main' into fix/app-partition-parallel-jobs 2026-08-17 10:22:20 +08:00
chichuan 36877d00dc Merge pull request #1024 from DingTalk-Real-AI/perf/schema-json-projection
perf: skip redundant JSON validation when projecting typed Schema values
2026-08-17 10:21:48 +08:00
xiatian a9a97c2746 Merge remote-tracking branch 'upstream/main' into codex/sheet-dropdown-source-range 2026-08-17 09:43:54 +08:00
RuiGong01 f72979f4a9 Merge branch 'main' into dws_0814_1723 2026-08-17 09:41:16 +08:00
chichuan 55d94d3b58 perf: skip redundant JSON validation when projecting typed Schema values
typedJSONValue marshaled a typed value and then routed the result through
rawJSONValue, which runs json.Valid before decoding. On that path the input is
whatever json.Marshal has just produced, so the validation scan can only ever
succeed: it re-read every marshaled document for nothing.

The decode step is now shared by both entry points. rawJSONValue keeps its
json.Valid check, because it still accepts untrusted input, while typedJSONValue
decodes what it marshaled directly. Across the 1121-tool set this removes about a
third of the Schema Catalog projection work: the internal/app schema suite goes
from 26.0s to 17.2s uninstrumented, and from 291.1s to 241.0s under -race.

The delivered Catalog is byte-for-byte unchanged. check-generated-drift,
check-schema-catalog and check-schema-binary each regenerate the same
source_hash sha256:93b8d44eb163bd2898c78397d22af92d378e3dc4e20f56b33277b51e4342e2e6,
and the two error contracts are preserved: typedJSONValue still rejects a value
json.Marshal cannot encode, and rawJSONValue still rejects invalid JSON.
2026-08-16 22:21:16 +08:00
chichuan bfd0976b31 fix(ci): run the app test partitions as parallel shards
The five internal/app partitions ran end to end inside one job, so the app
shard's wall clock was the sum of all five: 780s in CI, of which the schema
partition owned 357s. Each partition is now its own matrix shard, so they run
concurrently and the shard's wall clock is set by its slowest partition rather
than by their total. Every partition shard still selects the same single
internal/app package, so the impacted-package query maps the shard name back to
app and the partition only chooses which tests run.

The helper gains a partition argument and a list-partitions mode. APP_PARTITIONS
is the single source of truth for the set, and the discovery pass still runs in
every job, so each one independently verifies that the partition patterns cover
every top-level test exactly once before running the one it was asked for.

Two fail-closed checks guard the split, because the helper's own coverage check
can no longer prove the whole package ran once the partitions are separate jobs:

- The helper cross-checks APP_PARTITIONS against the coverage counters in both
  directions, so a counted partition that nothing dispatches and a dispatchable
  partition with no counter both fail instead of silently skipping tests.
- TestCIAppRacePartitionMatrixMatchesHelper pins the workflow's app-<partition>
  shards to list-partitions output in both directions, so a partition cannot
  lose its job while every job stays green.

The discovery loop variable is renamed from partition to spec: it would
otherwise shadow the partition requested on the command line, which run mode
reads after the discovery pass completes.
2026-08-16 22:18:04 +08:00
chichuan 4a33e7e893 fix(ci): drop race instrumentation from the app schema partition
The schema partition's 52 tests assert structural Schema-to-Cobra contracts over
a single goroutine: none of them call t.Parallel or start a goroutine, so the
race detector has no concurrent access to observe there. The process-global lazy
metadata that does need race coverage (schema_source_root's atomic.Value, the
parameter-binding lazy loaders) is exercised by internal/cli's concurrent tests,
which stay instrumented.

The instrumentation was not free here. The partition shares a single sync.Once
Catalog build whose work is allocation-heavy, and -race made it roughly 11x
slower: 26s -> 291s locally, and 357s of the app shard's 780s in CI. Within that
partition TestFinalSchemaToolsHaveExecutableBaseCommands alone accounted for
262s, not because the test is expensive but because it is the first caller to pay
for the shared snapshot; its 1121 subtests together measure 0.00s.

run_partition now takes the instrumentation mode explicitly and fails closed on
an unrecognized value, so a typo cannot silently drop -race from a partition that
is supposed to carry it.
2026-08-16 22:17:14 +08:00
github-actions[bot] ee74765383 Merge pull request #1019 from DingTalk-Real-AI/feat/help-feedback-entry
feat: add feedback survey entry to root help
2026-08-16 08:09:01 +08:00
玉澜 2855f1a84b fix(skills): multi-turn append semantics for explicit write requests
Eval forensics: in clean isolated doc-domain runs the toolsearch arm lost
7/20 paired cases to main on codex+gpt-5.6-sol; 5 of 6 flips showed the
same signature — the agent refused an explicit per-turn 'append/insert'
request because similar content already existed (front-loaded or matched),
and the judge fails the refusal. The clause makes per-turn explicit write
requests execute (existence may be noted but never replaces execution),
forbids front-loading future turns, and keeps read-back for verification
only. Verified: doc domain 30% -> 55% (main 60%), dup-refusal signature
5/6 -> 0.
2026-08-15 18:25:07 +08:00
xiatian 9fbd8addbe Merge remote-tracking branch 'upstream/main' into codex/sheet-dropdown-source-range 2026-08-15 13:40:55 +08:00
xiatian 92195a58a3 fix(sheet): align SourceRange review contract 2026-08-15 13:40:47 +08:00
玉澜 6406c93eaa feat(cli): convergence signals for capability-absent searches
Eval-driven optimizations from the 2026-08-14/15 agent evaluations:

- abstained lexical responses now carry a bounded hint naming the one
  permitted verification step (dws schema <product> --compact) and
  licensing the conclusion 'unsupported', so agents stop enumerating
  every product to prove a capability is missing
- responses whose candidates only matched weak fields (parameters/
  description, no identity/summary hit) set weak_match=true with a
  matching hint; the flag is field-class based, not a score threshold,
  preserving the RFC ban on absolute score cutoffs
- exact_filtered keeps its typed refusal without inheriting either hint;
  budget-truncation to zero candidates downgrades to the abstain hint
- dingtalk-shared gains the matching convergence rule: one search, at
  most one schema <product> verification, then conclude honestly

Evidence: the nonexistent-capability intent scenario took 15 turns/177s
to conclude 'no poll capability' with identical correctness to a 4-turn
run; these signals bound that path. Note: on the real catalog the poll
phrasing still matches summary fields of generic chat tools, so the
behavioral fix for that specific case rides on the shared-skill rule;
the wire flags cover genuinely unmatched and weak-field-only queries.
2026-08-15 05:02:28 +08:00
RuiGong01 0b012788c7 Merge branch 'main' into dws_0814_1723 2026-08-14 20:15:22 +08:00
玉澜 747a17f305 fix(cli): close tool search review findings from contract probe
- effect filter: case-insensitive against the reviewed vocabulary with a
  typed invalid_effect refusal; unknown product ids get a typed
  unknown_product refusal instead of a silent abstain (probe-confirmed:
  --effect Read used to return 0 candidates silently)
- DWS_TOOL_SEARCH_* unparsable values now warn on stderr while keeping
  the resolved default, so ranking experiments cannot silently measure
  defaults (probe-confirmed: K1='0,9' was ignored with empty stderr)
- --explain help text now declares it a dev-only diagnostic that exposes
  raw ordering scores; the default wire stays score-free
- lexical ordering quantizes the sort key at 1e-9 so architecture
  dependent FMA fusion (arm64 vs amd64) cannot flip near-tied candidates
  past the canonical tie-break; reported scores are unchanged

Contract probe re-run against this build: 11/11 checks OK (was 3 FINDINGs)
2026-08-14 19:58:09 +08:00
玉澜 d578458315 fix(cli): fail closed on unknown tool-search filters and warn on unparsable env
- effect filters are case-normalized against the reviewed vocabulary and
  reject anything else with a typed invalid_effect error instead of a
  silent abstain (probe-confirmed: --effect Read now equals --effect read)
- unknown product ids reject with unknown_product instead of returning
  abstained=true, so caller typos cannot masquerade as empty results
- unparsable DWS_TOOL_SEARCH_* float env values now warn on stderr;
  previously a typo like K1=0,9 silently measured the default config
- --explain help text declares the raw-score disclosure as a dev-only
  diagnostic exception to the score-free public wire
2026-08-14 19:53:34 +08:00
炳昱 e742a6c269 Merge remote-tracking branch 'upstream/main' into codex/fix-windows-event-bus 2026-08-14 19:40:58 +08:00
chichuan 05868610f0 Merge branch 'main' into codex/sheet-dropdown-source-range 2026-08-14 18:33:24 +08:00
炳昱 22649e96ef test(event): cover Unix spawn validation on Windows 2026-08-14 18:11:42 +08:00
炳昱 f68a11f11d test(event): cover Windows lifecycle edges 2026-08-14 18:05:34 +08:00
炳昱 abe5129306 fix(event): support Windows bus lifecycle 2026-08-14 17:56:51 +08:00
chichuan 7b7bd556e9 Merge branch 'main' into feat/calendar-event-share-info 2026-08-14 17:43:49 +08:00
ruigong d534ee242c fix(skill): scope doc/drive descriptions to entity-content vs file management 2026-08-14 17:33:54 +08:00
fengbai 90473284b8 fix(calendar): remove shell comment from share-info example
- Move the eventId lookup hint into Long description
- Keep example commands free of shell comments to pass example policy gate
2026-08-14 15:51:20 +08:00
玉澜 575b8f32c4 feat(skills): add compact cross-product read summary 2026-08-14 15:37:13 +08:00
fengbai 07aa2c883a fix(calendar): address CR comments for share-info
- Fix Example indentation (tab -> 2 spaces)
- Remove unsubstantiated default en-US from --language help/docs
- Add test asserting calendarId/language are omitted when only --id is passed
2026-08-14 15:10:20 +08:00
玉澜 70e46cbd29 fix(cli): harden tool search evaluation and config 2026-08-14 10:04:53 +08:00
xiatian 8cd2b0259d Merge remote-tracking branch 'upstream/main' into codex/sheet-dropdown-source-range 2026-08-13 18:04:51 +08:00
xiatian 4b8d94c24e ci: retry interrupted app race shard 2026-08-13 17:15:22 +08:00
玉澜 e9efd3e909 docs(rfc): sync progressive discovery numbers to post-hardening baseline 2026-08-13 16:58:31 +08:00
xiatian 76e5a8c4d9 Merge remote-tracking branch 'upstream/main' into codex/sheet-dropdown-source-range 2026-08-13 16:37:12 +08:00
玉澜 9bc803fde0 fix(cli): enforce shortcut avoid_when boilerplate filter and sync eval docs
Self-review follow-ups on the ranking hardening series:

- the avoid_when boilerplate filter compared lowercased phrases against
  a mixed-case constant, so the "需要该 Shortcut..." noise filter never
  fired; the constant is now lowercase (no ranking change: the boilerplate
  was never contained by any query, sentinel 990/1123 unchanged)
- add direct unit coverage for the avoid_when penalty (echoed-phrase
  match, unrelated-query pass-through, boilerplate filter, partial
  overlap, explain annotation)
- update tool-search-ranking-evaluation.md to the post-hardening
  numbers (ensemble R@5 88.16%, Forbidden 0%/0%, action_v1 shadow
  84.24% with the trade-off recorded)
- drop the dead algorithm_freeze manifest block; nothing consumes it
  after the hash decoupling
2026-08-13 16:31:52 +08:00
xiatian 6abffce4e5 fix(sheet): preserve dropdown schema compatibility 2026-08-13 16:13:30 +08:00
玉澜 2dc3147cda refactor(scripts): decouple tool search eval manifest from source hashes and seal gate
- drop the algorithm/gate source SHA256 freeze: every ranking change
  required a manual manifest hash sync, coupling code evolution to the
  manifest
- drop the --require-sealed release gate (sealed qrels hash, dual
  signatures, coverage minima, independent-result thresholds) and its
  tool-search-independent-data-gate Makefile target
- behavior regression protection stays with the Go evaluation sentinels
  that run the real assembled catalog
2026-08-13 16:09:19 +08:00
xiatian 86b78e45d7 Merge remote-tracking branch 'upstream/main' into codex/sheet-dropdown-source-range 2026-08-13 16:08:28 +08:00
玉澜 e91b30c769 chore(testdata): refresh frozen tool search algorithm source hash
The algorithm freeze gate intentionally rejects silent ranking source
changes; this records the new source set hash after the ranking hardening
commit.
2026-08-13 15:51:27 +08:00
玉澜 0df293186a feat(cli): harden tool search ranking, guardrails and observability
- default engine: fielded BM25 ensemble (action_v1 remains a shadow candidate)
- add --explain score_breakdown: per-field contributions, structured
  multiplier, query classification and avoid_when penalty
- add DWS_TOOL_SEARCH_* env overrides (algorithm, k1/b, field weights,
  explain) for release-independent tuning and rollback
- tokenizer: CJK unigram+bigram so single-character intents match; align
  isToolSearchCJK with unicode.Han used by the evaluation slices
- structured routing: extend vocabularies with English synonyms,
  word-boundary matching for ASCII values, restrict the rerank gate to
  technical identifiers only
- avoid_when demotion layer wraps every lexical algorithm (Forbidden@5
  proxy 37.55% -> 0); intent-overlap matching rejected after control
  showed it demoted gold answers (Recall@5 0.8664 -> 0.7845)
- remove the OA task_id +1.00 fixture-specific multiplier
- build the default engine once per process for long-lived hosts
- evaluation baseline: intent-proxy R@5 990/1123 (88.16%); update
  integration sentinels accordingly
2026-08-13 15:20:09 +08:00
玉澜 003bbe87ba Merge remote-tracking branch 'upstream/main' into test/tool-search-ranking-eval 2026-08-13 14:04:53 +08:00
xiatian 5065e4bfb6 Merge remote-tracking branch 'upstream/main' into codex/sheet-dropdown-source-range 2026-08-13 13:49:52 +08:00
玉澜 fc47e05cd7 docs: clarify local tool search boundaries 2026-08-13 13:21:28 +08:00
玉澜 86ebdd8f25 refactor(cli): remove tool search provider fallback 2026-08-13 11:51:33 +08:00
xiatian 2778bef5bd feat(sheet): support source range dropdowns and read completion 2026-08-13 11:46:58 +08:00
玉澜 20b8120f59 Merge remote-tracking branch 'upstream/main' into test/tool-search-ranking-eval 2026-08-13 11:10:45 +08:00
玉澜 a08a1f54fd docs(skill): integrate DWS command discovery 2026-08-13 11:09:31 +08:00
玉澜 93da35413b feat(cli): add progressive tool search 2026-08-13 10:15:10 +08:00
fengbai 8aee08268d test(calendar): add event share-info dry-run and required-flag tests 2026-08-12 21:17:32 +08:00
fengbai 6a4744073c feat(calendar): add event share-info command 2026-08-12 21:07:59 +08:00
151 changed files with 16082 additions and 440 deletions
@@ -0,0 +1,11 @@
---
category: Fixed
---
- **Aitable pagination and Minutes unshare verification** (#1006) — keeps
record queries on the service's 20-record page boundary so multi-page reads
and mutation readbacks no longer report false retryable failures, preserves
`totalCount` when supplied, validates `--dry-run` plans before transport,
follows active deletion readback continuations before proving absence, and
rejects Minutes unshare success until the listening note exists and the
service acknowledges the exact task and member targets.
+5
View File
@@ -0,0 +1,5 @@
---
category: Added
---
- **Robot group reference replies** (#928) — `chat message send-by-bot` supports paired `--reply` and `--ref-sender` flags for Markdown replies that quote an existing group message.
@@ -0,0 +1,5 @@
---
category: Fixed
---
- **Document write verification** (#960) — avoids false partial-success results when normalized Markdown, paginated blocks, inline images, or version reverts are confirmed by server readback. Document reverts and media inserts now require explicit readback evidence and report partial success when the server cannot prove the requested result.
+5
View File
@@ -0,0 +1,5 @@
---
category: Added
---
- **Calendar event share-info** (#980) — adds `dws calendar event share-info` to fetch a calendar event's share info (title, organizer, location, join info) for sharing with others; supports `--calendar-id` and `--language`.
@@ -0,0 +1,5 @@
---
category: Fixed
---
- **Chat sender identity guards** — preserves unverified mixed sender inputs after exact message `senderId` matches and aligns `--sender-query` Skill guidance with fail-closed Runtime behavior.
+5
View File
@@ -0,0 +1,5 @@
---
category: Changed
---
- **Doc/drive description scope** — restates the `dingtalk-doc` description as document-entity-and-content operations with an explicit exclusion list, and narrows `dingtalk-drive` to file-level management of DingTalk documents, so first-round Agent selection separates content work from file management without changing CLI behavior.
@@ -0,0 +1,8 @@
---
category: Changed
---
- **Faster Schema Catalog assembly** — projects typed values into payload JSON
without re-running a validation scan over documents `json.Marshal` has just
produced, cutting roughly a third of the projection work across the full tool
set. Untrusted JSON input keeps its existing validation.
+6
View File
@@ -0,0 +1,6 @@
---
category: Added
---
- **Sheet SourceRange dropdowns** — supports range-backed dropdowns across direct, cell, and batch write paths, with structured readback for valid and invalid references. Batch `set-dropdown` now rejects unsupported top-level `colors` / `source-colors`; Inline colors belong in `options[].color`, while SourceRange color writes remain unsupported.
- **Sheet read completion metadata** — documents and preserves returned ranges, truncation reasons, and partial-read status for large range and CSV reads.
+5
View File
@@ -0,0 +1,5 @@
---
category: Fixed
---
- **Windows event bus lifecycle** — start event consumers without unsupported inherited file descriptors, stop buses through local IPC with a termination fallback, and preserve subscription cleanup when startup fails.
+63 -22
View File
@@ -523,12 +523,22 @@ jobs:
# shards at full-suite scope; release-scripts is included because its
# dedicated job only runs at full-suite or release-sensitive scope, and
# dropping it here would stop testing test/scripts changes entirely.
# internal/app is carried by one shard per bounded partition rather than a
# single app shard: the partitions used to run end to end inside one job,
# where the Schema partition alone owned most of the wall clock. The
# app-<partition> names are pinned to the helper's partition set by
# TestCIAppRacePartitionMatrixMatchesHelper, so a partition can never lose
# its job silently.
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shard:
- app
- app-schema
- app-a-b
- app-c
- app-d-r
- app-s-z-example-fuzz
- generators
- helpers
- cli
@@ -570,9 +580,16 @@ jobs:
TEST_SHARD: ${{ matrix.shard }}
run: |
set -euo pipefail
# Every app partition shard tests the same single internal/app
# package, so the impacted-package query uses the base shard name and
# the partition only selects which tests run.
package_shard="$TEST_SHARD"
case "$TEST_SHARD" in
app-*) package_shard=app ;;
esac
package_output="$(
./scripts/ci/changed-test-packages.sh \
list-shard "$TEST_SHARD" "$TEST_BASE_REF" "$TEST_HEAD_REF"
list-shard "$package_shard" "$TEST_BASE_REF" "$TEST_HEAD_REF"
)"
if [ -z "$package_output" ]; then
echo "No buildable Go package in shard $TEST_SHARD is affected by this revision." \
@@ -612,14 +629,19 @@ jobs:
exit 1
}
done
if [ "$TEST_SHARD" = "app" ]; then
# A single long-lived app test process retains every constructed
# command tree in framework registries. Isolate Schema assembly and
# bounded name ranges so each process releases that state on exit.
test "${#packages[@]}" -eq 1
./scripts/ci/run-app-race-tests.sh run "${packages[0]}"
exit 0
fi
case "$TEST_SHARD" in
app-*)
# A single long-lived app test process retains every constructed
# command tree in framework registries. Each partition is its own
# job, so that state is released when the process exits and the
# partitions run concurrently instead of end to end. The helper
# still verifies that the partition patterns cover every top-level
# test exactly once before running the one it was asked for.
test "${#packages[@]}" -eq 1
./scripts/ci/run-app-race-tests.sh run "${packages[0]}" "${TEST_SHARD#app-}"
exit 0
;;
esac
if [ "$TEST_SHARD" = "release-scripts" ]; then
# Mirror the dedicated release-contract job: these suites shell out
# to archive tooling and are not race-instrumented there.
@@ -640,14 +662,21 @@ jobs:
needs: lint
if: ${{ needs.lint.outputs.changelog_only != 'true' && needs.lint.outputs.docs_only != 'true' && needs.lint.outputs.full_suite == 'true' }}
runs-on: ubuntu-latest
# app runs several independently bounded processes; cli/smoke need headroom
# beyond go test -timeout for setup + assembly.
# internal/app is split across one shard per bounded partition so the
# partitions run concurrently and each releases its framework registries
# when the process exits; cli/smoke need headroom beyond go test -timeout for
# setup + assembly. The app-<partition> names are pinned to the helper's
# partition set by TestCIAppRacePartitionMatrixMatchesHelper.
timeout-minutes: 20
strategy:
fail-fast: false
matrix:
shard:
- app
- app-schema
- app-a-b
- app-c
- app-d-r
- app-s-z-example-fuzz
- generators
- helpers
- cli
@@ -673,18 +702,30 @@ jobs:
TEST_SHARD: ${{ matrix.shard }}
run: |
set -euo pipefail
package_output="$(./scripts/ci/test-packages.sh list "$TEST_SHARD")"
# Every app partition shard tests the same single internal/app
# package, so the package query uses the base shard name and the
# partition only selects which tests run.
package_shard="$TEST_SHARD"
case "$TEST_SHARD" in
app-*) package_shard=app ;;
esac
package_output="$(./scripts/ci/test-packages.sh list "$package_shard")"
test -n "$package_output"
mapfile -t packages <<< "$package_output"
test "${#packages[@]}" -gt 0
if [ "$TEST_SHARD" = "app" ]; then
# A single long-lived app test process retains every constructed
# command tree in framework registries. Isolate Schema assembly and
# bounded name ranges so each process releases that state on exit.
test "${#packages[@]}" -eq 1
./scripts/ci/run-app-race-tests.sh run "${packages[0]}"
exit 0
fi
case "$TEST_SHARD" in
app-*)
# A single long-lived app test process retains every constructed
# command tree in framework registries. Each partition is its own
# job, so that state is released when the process exits and the
# partitions run concurrently instead of end to end. The helper
# still verifies that the partition patterns cover every top-level
# test exactly once before running the one it was asked for.
test "${#packages[@]}" -eq 1
./scripts/ci/run-app-race-tests.sh run "${packages[0]}" "${TEST_SHARD#app-}"
exit 0
;;
esac
# cli/smoke own heavy NewRootCommand / Schema assembly under -race;
# give them a dedicated package timeout on slower hosted runners.
timeout_budget=12m
+31
View File
@@ -6,6 +6,37 @@ The format is inspired by [Keep a Changelog](https://keepachangelog.com/) and th
## [Unreleased]
## [1.0.59-beta.2] - 2026-08-17
### Added
- **Privacy-safe CLI telemetry** (#1009) — reports reviewed command outcomes and profile identity dimensions while excluding command arguments, output, paths, device fingerprints, and automatic system dimensions; `DO_NOT_TRACK=1` disables reporting.
- **Feedback survey entry in root help** (#1019) — `dws --help` now closes with a Feedback section linking the user-experience survey form.
- **Wiki Shortcut workflows** — publishes 20 reviewed space, member, node, and
activity shortcuts with strict collection validation, cursor handling,
write-terminal evidence, safe read-backs where the backend supports them,
task-oriented routing, and documented backend
boundaries.
### Changed
- **Chat IM ID flags** (#954) — standardizes chat command entry points on `--conversation-id` for conversation IDs and `--message-id` for message IDs, so help, Schema, and Agent recommendations use the same canonical flags.
- **Legacy chat flag compatibility** (#954) — keeps older chat IM ID flags such as `--group`, `--id`, `--chat`, `--open-conversation-id`, `--msg-id`, and `--open-message-id` working as compatibility aliases where applicable, while hiding migrated aliases from recommended help and Schema surfaces.
- **Chat group bots target flag** (#954) — keeps `dws chat group bots` on the visible `--group` flag; this command does not register `--group-name`, and `--group` accepts either an openConversationId or a uniquely resolved group name.
- **Faster Schema Catalog assembly** — projects typed values into payload JSON
without re-running a validation scan over documents `json.Marshal` has just
produced, cutting roughly a third of the projection work across the full tool
set. Untrusted JSON input keeps its existing validation.
### Fixed
- **Chat card update evidence** — distinguishes an accepted update request from an independently verified visible update, preserving the real `bizId` and warning callers not to repeat an unverified write.
- **Chat command guidance** — splits message and group references by task and explains that `--from` is ambiguous between sender and time-range intent.
## [1.0.59-beta.1] - 2026-08-14
### Added
+11 -11
View File
@@ -1,33 +1,33 @@
class DingtalkWorkspaceCliBeta < Formula
desc "Automate DingTalk workspace tasks from the terminal (beta channel)"
homepage "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli"
version "1.0.59-beta.1"
version "1.0.59-beta.2"
license "Apache-2.0"
keg_only "it is the beta channel and conflicts with dingtalk-workspace-cli"
on_macos do
if Hardware::CPU.arm?
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.1/dws-darwin-arm64.tar.gz"
sha256 "36a30f3496e0f759c15c0b09f67dbd23b8ecdfff2eebe572f88125b26485830f"
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.2/dws-darwin-arm64.tar.gz"
sha256 "7f11218d3222f3e93c3b1e94b3a004c061eb0a297b206447ea95fe6b2b1ec674"
else
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.1/dws-darwin-amd64.tar.gz"
sha256 "e7a04906380efd8da88cd112e6a512bb6470a3956dc370150037ed6e314db445"
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.2/dws-darwin-amd64.tar.gz"
sha256 "72f06a334cf29d23123639fabe13bf2951765066136e47ccc6453667c382f8c2"
end
end
on_linux do
if Hardware::CPU.arm?
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.1/dws-linux-arm64.tar.gz"
sha256 "f59ab055f3e841e4cebc964ae3ef969668475548abaaf8bede44afdca9a3e28d"
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.2/dws-linux-arm64.tar.gz"
sha256 "3edbfabb7718b53914a2d9efe7b1152e9ebd70765ff0b6f19ad3125e4a2458ed"
else
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.1/dws-linux-amd64.tar.gz"
sha256 "2c8f919489d958c7d49262615e81faac70a9fbcae2d589ab54a0bb3c5700a057"
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.2/dws-linux-amd64.tar.gz"
sha256 "e1c610070a9c1b3763656cbd53c818290f199097adc07cb9fe629ed765c5e1e0"
end
end
resource "skills" do
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.1/dws-skills.zip"
sha256 "25f4a7e1d01fa4d771d79201b34b11ee8a24182bdcdc94bfb98d2bd5845bed3b"
url "https://github.com/DingTalk-Real-AI/dingtalk-workspace-cli/releases/download/v1.0.59-beta.2/dws-skills.zip"
sha256 "aa2854651eaa2c857b526aaec73fd1358d31b11999fe7fd3e99e5060b2522a1f"
end
def install
+23 -2
View File
@@ -10,7 +10,7 @@ SCHEMA_META_INDEX_OUTPUT ?= artifacts/schema_meta_index.gob
POLICY_ENV = DWS_POLICY_TMPDIR="$(DWS_POLICY_TMPDIR)" GOTMPDIR="$(POLICY_GOTMPDIR)"
GO_SOURCE_LIST = git ls-files -z --cached --others --exclude-standard -- '*.go'
.PHONY: all help build rebuild test test-plan test-auth-legacy-compat lint format-check fmt policy edition-test interface-integrity authoritative-interface-integrity coverage-gate coverage-gate-platform update-interface-baseline reset-interface-baseline schema-compatibility skill-command-integrity skill-context-budget multi-im-skill-chain-integrity cli-smoke mock-mcp-smoke test-schema-agent-examples generate-schema fetch-mcp-metadata generate-schema-catalog package release release-pre release-stable changelog-pre changelog-stable publish-homebrew-formula setup-hooks
.PHONY: all help build rebuild test test-plan test-auth-legacy-compat lint format-check fmt policy edition-test interface-integrity authoritative-interface-integrity coverage-gate coverage-gate-platform update-interface-baseline reset-interface-baseline schema-compatibility skill-command-integrity skill-context-budget multi-im-skill-chain-integrity cli-smoke mock-mcp-smoke test-schema-agent-examples tool-search-evaluation-harness generate-tool-search-comparison generate-schema fetch-mcp-metadata generate-schema-catalog package release release-pre release-stable changelog-pre changelog-stable publish-homebrew-formula setup-hooks
all: setup-hooks fmt lint build test rebuild
@@ -37,6 +37,8 @@ help:
@printf " make cli-smoke - Verify help for every public top-level command\n"
@printf " make mock-mcp-smoke - Verify HTTP and stdio MCP request/response transport\n"
@printf " make test-schema-agent-examples - Contract-check all Agent examples and dry-run the eligible subset\n"
@printf " make tool-search-evaluation-harness - Verify Go retrieval, trust metrics, frozen fixtures, and build-time report generation\n"
@printf " make generate-tool-search-comparison - Generate the declaration-assembled comparison under .worktrees/policy-tmp\n"
@printf " make generate-schema - Refresh param_aliases + verify Schema assembly determinism\n"
@printf " make generate-schema-catalog - Optional assembled Catalog dump under artifacts/ (not a delivery step)\n"
@printf " make package - Build all release artifacts locally\n"
@@ -84,7 +86,7 @@ fmt:
$(GO_SOURCE_LIST) > "$$go_files"; \
xargs -0 sh -c 'if [ "$$#" -gt 0 ]; then exec gofmt -w -- "$$@"; fi' sh < "$$go_files"
policy: test-auth-legacy-compat
policy: test-auth-legacy-compat tool-search-evaluation-harness skill-mono-multi-content
@mkdir -p "$(POLICY_GOTMPDIR)"
@$(POLICY_ENV) ./scripts/policy/check-open-source-assets.sh
@$(POLICY_ENV) ./scripts/policy/check-skill-context-budget.sh
@@ -161,6 +163,25 @@ mock-mcp-smoke:
test-schema-agent-examples:
DWS_AGENT_EXAMPLES_DRY_RUN=1 $(GO) test -v -count=1 ./internal/app -run '^TestAgentExamplesDryRun$$'
tool-search-evaluation-harness:
python3 -m unittest \
scripts/dev/eval_tool_search_ranking_test.py \
scripts/dev/verify_tool_search_eval_manifest_test.py
@mkdir -p "$(DWS_POLICY_TMPDIR)"
$(GO) run ./internal/generator/cmd_schema_catalog \
-root . \
-output "$(DWS_POLICY_TMPDIR)/tool-search-schema-catalog" \
-meta-index "$(DWS_POLICY_TMPDIR)/tool-search-schema-meta-index.gob"
python3 scripts/dev/verify_tool_search_eval_manifest.py \
--catalog-dir "$(DWS_POLICY_TMPDIR)/tool-search-schema-catalog"
$(GO) test ./internal/cli -run '^TestToolSearch|^TestSchemaSearch|^TestDefaultToolSearch|^TestAggregateToolSearch|^TestScoreToolSearch' -count=1
$(GO) test ./internal/app -run '^TestToolSearchDelivery|^TestSchemaSearchInspectCatalogVersionContract$$' -count=1
$(GO) run ./internal/generator/cmd_tool_search_comparison -output "$(DWS_POLICY_TMPDIR)/tool-search-comparison.json"
generate-tool-search-comparison:
@mkdir -p "$(DWS_POLICY_TMPDIR)"
$(GO) run ./internal/generator/cmd_tool_search_comparison -output "$(DWS_POLICY_TMPDIR)/tool-search-comparison.json"
# generate-schema refreshes param_aliases_generated.go and verifies that
# ResolveSchemaBuild assembly is deterministic. Catalog is runtime-assembled
# (声明即 Catalog); cmd_schema_catalog is not a committed delivery step.
File diff suppressed because it is too large Load Diff
+1
View File
@@ -18,6 +18,7 @@ require (
github.com/muesli/termenv v0.16.0
github.com/open-dingtalk/dingtalk-stream-sdk-go v0.9.2-beta.1
github.com/spf13/cobra v1.10.2
github.com/yuin/goldmark v1.8.5
github.com/zalando/go-keyring v0.2.8
gitlab.alibaba-inc.com/aes/aem-go-sdk v0.3.0
golang.org/x/crypto v0.49.0
+2
View File
@@ -105,6 +105,8 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e h1:JVG44RsyaB9T2KIHavMF/ppJZNG9ZpyihvCd0w101no=
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e/go.mod h1:RbqR21r5mrJuqunuUZ/Dhy/avygyECGrLceyNeo4LiM=
github.com/yuin/goldmark v1.8.5 h1:r6N5afV5qj/5S4UTch8agZHJ8UxNCMwX7WjkkJam2NA=
github.com/yuin/goldmark v1.8.5/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/zalando/go-keyring v0.2.8 h1:6sD/Ucpl7jNq10rM2pgqTs0sZ9V3qMrqfIIy5YPccHs=
github.com/zalando/go-keyring v0.2.8/go.mod h1:tsMo+VpRq5NGyKfxoBVjCuMrG47yj8cmakZDO5QGii0=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
+2 -1
View File
@@ -1220,7 +1220,8 @@ func newEventStopCommandWithFlags(globalFlags ...*GlobalFlags) *cobra.Command {
editionName := editionNameOrDefault()
clientIDHash := dwsevent.ClientIDHash(clientID)
workDir := eventWorkDir(configDir, editionName, dwsevent.SourceKindAppStream, clientIDHash)
if err := eventStopBus(busctl.StopConfig{WorkDir: workDir}); err != nil {
ipcEndpoint := defaultIPCEndpoint(workDir, editionName, dwsevent.SourceKindAppStream, clientIDHash)
if err := eventStopBus(busctl.StopConfig{WorkDir: workDir, IPCEndpoint: ipcEndpoint}); err != nil {
if errors.Is(err, busctl.ErrNotRunning) {
fmt.Fprintln(c.OutOrStdout(), "bus is not running")
return nil
+1 -1
View File
@@ -1220,7 +1220,7 @@ func runPersonalEventStop(c *cobra.Command, opts personalStopOptions) error {
}
busState := "personal bus stopped"
if err := personalStopBus(busctl.StopConfig{WorkDir: workDir}); err != nil {
if err := personalStopBus(busctl.StopConfig{WorkDir: workDir, IPCEndpoint: ipcEndpoint}); err != nil {
if errors.Is(err, busctl.ErrNotRunning) {
busState = "personal bus is not running"
} else {
+3 -1
View File
@@ -52,6 +52,8 @@ func (c *paramAliasCaptureCaller) paramAliasResponseForTool(tool string) string
switch tool {
case "list_calendar_events":
return `{"result":{"events":[]}}`
case "query_records":
return `{"success":true,"status":"success","error":{},"data":{}}`
case "search_mail_users":
return `{"users":[{"name":"Fixture User","email":"fixture@example.com","id":"fixture-user"}]}`
case "search_dept_by_keyword":
@@ -63,7 +65,7 @@ func (c *paramAliasCaptureCaller) paramAliasResponseForTool(tool string) string
case "list_doc_versions":
return `{"result":{"items":[{"version":3}]}}`
case "revert_doc_version":
return `{"version":3}`
return `{"revertedToVersion":3}`
case "search_doc_templates":
return `{"result":[{"templateId":"fixture-template-id"}]}`
case "create_document":
+47
View File
@@ -462,6 +462,53 @@ func TestCalendarEventListDryRunPreviewsOnly(t *testing.T) {
}
}
func TestCalendarEventShareInfoDryRunPreviewsOnly(t *testing.T) {
got, err := executeRootCaptureStdout(t, []string{
"--dry-run", "calendar", "event", "share-info",
"--id", "EVT_001",
"--language", "zh-CN",
"--calendar-id", "primary",
})
if err != nil {
t.Fatalf("calendar event share-info --dry-run error = %v\n%s", err, got)
}
for _, want := range []string{"get_event_share_info", "eventId", "EVT_001", "zh-CN", "primary"} {
if !strings.Contains(got, want) {
t.Fatalf("calendar event share-info dry-run output missing %q:\n%s", want, got)
}
}
}
func TestCalendarEventShareInfoRequiresEventID(t *testing.T) {
got, err := executeRootCaptureStdout(t, []string{
"--dry-run", "calendar", "event", "share-info",
})
if err == nil {
t.Fatalf("calendar event share-info without --id: expected error, got nil\n%s", got)
}
if strings.Contains(got, "\"executed\": true") {
t.Fatalf("share-info without --id must not execute:\n%s", got)
}
}
func TestCalendarEventShareInfoOmitsOptionalArgs(t *testing.T) {
got, err := executeRootCaptureStdout(t, []string{
"--dry-run", "calendar", "event", "share-info",
"--id", "EVT_001",
})
if err != nil {
t.Fatalf("calendar event share-info --dry-run with only --id error = %v\n%s", err, got)
}
if !strings.Contains(got, "\"eventId\"") {
t.Fatalf("calendar event share-info dry-run output missing eventId:\n%s", got)
}
for _, unwanted := range []string{"\"calendarId\"", "\"language\""} {
if strings.Contains(got, unwanted) {
t.Fatalf("calendar event share-info dry-run with only --id should not contain %q:\n%s", unwanted, got)
}
}
}
func TestRootKeepsSVIPChatCompatibilityFlags(t *testing.T) {
root := NewRootCommand()
+14
View File
@@ -400,6 +400,7 @@ func schemaContractPayloadForBoundCanonicals(t *testing.T, root *cobra.Command,
func TestChatSchemaSeparatesSendAndReply(t *testing.T) {
snapshot := schemaContractPayloadForBoundCanonicals(t, NewRootCommand(),
"chat.send_personal_message",
"chat.send_robot_message",
"chat.reply_personal_message",
)
@@ -418,6 +419,19 @@ func TestChatSchemaSeparatesSendAndReply(t *testing.T) {
if _, exists := snapshot.Tools["chat.upload_conversation_file"]; exists {
t.Fatal("downlined chat file upload must not be advertised in Schema")
}
botReply := snapshot.Tools["chat.send_robot_message"]
botParams := schemaContractMap(botReply["parameters"])
if got := schemaContractString(botParams["reply"]["property"]); got != "referenceOpenMessageId" {
t.Fatalf("bot --reply property = %q", got)
}
if got := schemaContractString(botParams["ref-sender"]["property"]); got != "srcMsgSendOpenDingTalkId" {
t.Fatalf("bot --ref-sender property = %q", got)
}
if got, ok := botParams["title"]["required"].(bool); !ok || got {
t.Fatalf("bot --title required = %#v, want false for conditional Markdown input", botParams["title"]["required"])
}
assertSchemaContractConstraintGroup(t, botReply, "require_together", []string{"reply", "ref-sender"})
}
func TestCalendarAttendeeDeleteSchemaMatchesRuntimeGate(t *testing.T) {
@@ -0,0 +1,249 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
package app
import (
"bytes"
"context"
"encoding/json"
"errors"
"strings"
"testing"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cli"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
)
func TestToolSearchDeliveryActionSiblingRouting(t *testing.T) {
registerSchemaRuntimeDelivery()
// This test pins the action_v1 structured sibling routing itself, decoupled
// from the production default (now the fielded BM25 ensemble, which has no
// structured multiplier and cannot right-size these sibling queries).
config := cli.DefaultToolSearchConfig()
config.LexicalAlgorithm = cli.ToolSearchLexicalBM25Action
engine, err := cli.NewDeliveryToolSearchEngineWithConfig(config)
if err != nil {
t.Fatalf("NewDeliveryToolSearchEngineWithConfig() error = %v", err)
}
tests := []struct {
query string
gold string
}{
{query: "创建一场新的会议日程", gold: "calendar.create_calendar_event"},
{query: "给钉盘文件添加同事阅读权限", gold: "drive.add_permission"},
// The new main Catalog declares shortcut_fetch as a reviewed equivalent
// read surface, so this query has graded-equivalent Top-1 answers.
{query: "读取钉钉在线文字文档正文", gold: "doc.shortcut_fetch|doc.get_document_content"},
{query: "查询当前用户待审批任务 ID", gold: "oa.list_pending_tasks"},
{query: "同意指定审批任务", gold: "oa.approve_processInstance"},
{query: "按群名关键词搜索群聊", gold: "chat.search_groups"},
}
for _, test := range tests {
t.Run(test.gold, func(t *testing.T) {
response, searchErr := engine.Search(context.Background(), cli.ToolSearchRequest{Query: test.query, Limit: 5})
if searchErr != nil {
t.Fatalf("Search() error = %v", searchErr)
}
if len(response.Candidates) == 0 || !containsCanonicalAlternative(test.gold, response.Candidates[0].CanonicalPath) {
t.Fatalf("Search(%q) candidates = %#v, want %s first", test.query, response.Candidates, test.gold)
}
})
}
}
func containsCanonicalAlternative(alternatives, candidate string) bool {
for _, alternative := range strings.Split(alternatives, "|") {
if candidate == alternative {
return true
}
}
return false
}
func TestToolSearchDeliveryDiagnosticTrustAndChineseSlices(t *testing.T) {
registerSchemaRuntimeDelivery()
workflows := []cli.ToolSearchWorkflowEvaluationCase{{
ID: "chat-send-file-and-check-read",
Query: "给群里发文件,并确认群成员是否已读",
Subqueries: []string{"以当前用户身份给群聊发送本地文件", "查询指定消息的已读状态和人员"},
Required: []string{"chat.send_personal_message", "chat.query_msg_read_status"},
}}
report, err := cli.BuildDeliveryToolSearchDiagnosticComparison(context.Background(), workflows)
if err != nil {
t.Fatalf("BuildDeliveryToolSearchDiagnosticComparison() error = %v", err)
}
// Do not pin absolute Catalog-size counts here: they go stale when main
// adds tools. Enforce the size-independent trust contract below; observed
// counts are logged for CI forensics and live in the generated comparison
// JSON (make generate-tool-search-comparison).
t.Logf("intent proxy = %+v excluded_over_budget=%d", report.IntentProxy, report.IntentExcludedOverBudget)
t.Logf("language slices = %+v", report.IntentLanguageSlices)
t.Logf("context comparison = %+v", report.Context)
if report.IntentProxy.Cases == 0 {
t.Fatal("intent proxy population is empty; the diagnostic set did not build")
}
if report.IntentProxy.ZeroResultRate != 0 {
t.Fatalf("intent proxy zero-result rate = %v, want every reviewed intent to retrieve something", report.IntentProxy.ZeroResultRate)
}
for _, slice := range []string{"chinese_only", "mixed_chinese_ascii"} {
if report.IntentLanguageSlices[slice].Cases == 0 {
t.Fatalf("language slice %q is empty: %+v", slice, report.IntentLanguageSlices)
}
}
identity := report.Trust.Identity
// Identity coverage must stay exhaustive over the whole Catalog. Deriving the
// expected count from the same report keeps that contract enforced without
// hardcoding a tool count.
if identity.CanonicalCases != report.Context.ToolCount || identity.NFKCCases != report.Context.ToolCount {
t.Fatalf("identity coverage = %+v, want one case per %d Catalog tools", identity, report.Context.ToolCount)
}
if identity.CanonicalPassRate != 1 || identity.PrimaryCLIPassRate != 1 ||
identity.AliasCases == 0 || identity.AliasPassRate != 1 || identity.NFKCPassRate != 1 ||
identity.ExactFilteredPassRate != 1 {
t.Fatalf("identity trust = %+v", identity)
}
integrity := report.Trust.Integrity
if integrity.CatalogBindingFailures != 0 || integrity.UnknownCandidateCount != 0 || integrity.IneligibleCandidateCount != 0 || integrity.ResponseBudgetViolations != 0 {
t.Fatalf("integrity violations = %+v", integrity)
}
if report.Context.ToolCount == 0 || report.Context.ReductionVsFullSchema < 0.99 {
t.Fatalf("context comparison = %+v", report.Context)
}
}
func TestSchemaSearchInspectCatalogVersionContract(t *testing.T) {
registerSchemaRuntimeDelivery()
search := cli.NewSchemaCommand()
var searchOutput bytes.Buffer
search.SetOut(&searchOutput)
search.SetArgs([]string{"search", "--query", "chat.query_msg_read_status"})
if err := search.Execute(); err != nil {
t.Fatalf("schema search execute: %v", err)
}
var response cli.ToolSearchResponse
if err := json.Unmarshal(searchOutput.Bytes(), &response); err != nil {
t.Fatalf("decode search response: %v", err)
}
if response.Strategy != "exact_guard" || len(response.Candidates) != 1 || response.Candidates[0].CanonicalPath != "chat.query_msg_read_status" {
t.Fatalf("search response = %#v", response)
}
inspect := cli.NewSchemaCommand()
var inspectOutput bytes.Buffer
inspect.SetOut(&inspectOutput)
inspect.SetArgs([]string{
"chat.query_msg_read_status", "--compact",
"--expected-source-hash", response.Catalog.SourceHash,
"--expected-surface-hash", response.Catalog.SurfaceHash,
})
if err := inspect.Execute(); err != nil {
t.Fatalf("schema inspect execute: %v", err)
}
var payload cli.SchemaInspectV1Response
if err := json.Unmarshal(inspectOutput.Bytes(), &payload); err != nil {
t.Fatalf("decode inspect response: %v", err)
}
if payload.Version != "schema-inspect.v1" || payload.Catalog != response.Catalog || payload.ToolSpec["canonical_path"] != "chat.query_msg_read_status" {
t.Fatalf("inspect envelope = %#v", payload)
}
stale := cli.NewSchemaCommand()
stale.SilenceErrors = true
stale.SilenceUsage = true
stale.SetArgs([]string{
"chat.query_msg_read_status", "--compact",
"--expected-source-hash", "sha256:stale",
"--expected-surface-hash", response.Catalog.SurfaceHash,
})
err := stale.Execute()
var typed *apperrors.Error
if !errors.As(err, &typed) || typed.Reason != "catalog_changed" || typed.Category != apperrors.CategoryDiscovery {
t.Fatalf("stale inspect error = %#v", err)
}
incomplete := cli.NewSchemaCommand()
incomplete.SilenceErrors = true
incomplete.SilenceUsage = true
incomplete.SetArgs([]string{
"chat.query_msg_read_status",
"--expected-source-hash", response.Catalog.SourceHash,
})
err = incomplete.Execute()
if !errors.As(err, &typed) || typed.Category != apperrors.CategoryValidation {
t.Fatalf("incomplete inspect version error = %#v", err)
}
}
func TestSchemaSearchRejectsIgnoredGlobalOutputFlags(t *testing.T) {
for _, args := range [][]string{
{"schema", "search", "--query", "查询群消息已读状态", "--fields", "canonical_path"},
{"schema", "search", "--query", "查询群消息已读状态", "--jq", ".candidates"},
{"schema", "search", "--query", "查询群消息已读状态", "--format", "pretty"},
} {
root := NewRootCommand()
root.SilenceErrors = true
root.SilenceUsage = true
root.SetArgs(args)
err := root.Execute()
var typed *apperrors.Error
if !errors.As(err, &typed) || typed.Category != apperrors.CategoryValidation {
t.Fatalf("args %v error = %#v, want validation", args, err)
}
}
}
// BenchmarkToolSearchDeliveryChineseQuery measures the shipped search path
// against the complete declaration-assembled Catalog. The smaller benchmark
// in internal/cli intentionally uses a four-tool fixture and must not be used
// as the release-scale latency or allocation number.
func BenchmarkToolSearchDeliveryChineseQuery(b *testing.B) {
registerSchemaRuntimeDelivery()
engine, err := cli.NewDeliveryToolSearchEngineWithConfig(cli.DefaultToolSearchConfig())
if err != nil {
b.Fatalf("NewDeliveryToolSearchEngineWithConfig() error = %v", err)
}
request := cli.ToolSearchRequest{Query: "给群里发文件并确认消息已读", Limit: 5, CandidateLimit: 20}
b.ReportAllocs()
b.ResetTimer()
for index := 0; index < b.N; index++ {
if _, searchErr := engine.Search(context.Background(), request); searchErr != nil {
b.Fatalf("Search() error = %v", searchErr)
}
}
}
func BenchmarkToolSearchDeliveryDecomposedWorkflow(b *testing.B) {
registerSchemaRuntimeDelivery()
engine, err := cli.NewDeliveryToolSearchEngineWithConfig(cli.DefaultToolSearchConfig())
if err != nil {
b.Fatalf("NewDeliveryToolSearchEngineWithConfig() error = %v", err)
}
request := cli.ToolSearchRequest{Query: "给群里发文件,并确认群成员是否已读", Limit: 5, CandidateLimit: 20}
subqueries := []string{"以当前用户身份给群聊发送本地文件", "查询指定消息的已读状态和人员"}
b.ReportAllocs()
b.ResetTimer()
for index := 0; index < b.N; index++ {
if _, searchErr := engine.SearchSubqueries(context.Background(), subqueries, request); searchErr != nil {
b.Fatalf("SearchSubqueries() error = %v", searchErr)
}
}
}
func BenchmarkToolSearchDeliveryIndexBuild(b *testing.B) {
registerSchemaRuntimeDelivery()
config := cli.DefaultToolSearchConfig()
// Prime and cache declaration assembly so this benchmark isolates building
// the Tool Search projection/index. Cold process startup is measured at the
// release-binary boundary and reported separately.
if _, err := cli.NewDeliveryToolSearchEngineWithConfig(config); err != nil {
b.Fatalf("prime delivery Catalog: %v", err)
}
b.ReportAllocs()
b.ResetTimer()
for index := 0; index < b.N; index++ {
if _, err := cli.NewDeliveryToolSearchEngineWithConfig(config); err != nil {
b.Fatalf("NewDeliveryToolSearchEngineWithConfig() error = %v", err)
}
}
}
+53 -2
View File
@@ -23,6 +23,15 @@ import (
"github.com/spf13/cobra"
)
// SchemaInspectV1Response is the versioned Search -> Inspect wire contract.
// Legacy schema queries without expected hashes keep their historical flat
// payload for human and script compatibility.
type SchemaInspectV1Response struct {
Version string `json:"version"`
Catalog CatalogVersionRef `json:"catalog"`
ToolSpec map[string]any `json:"tool_spec"`
}
// schemaCommandCatalogError / payloads use deliverySchemaCatalog
// (RegisterSchemaSourceRoot → ResolveSchemaBuild). There is no committed
// Schema Catalog embed fallback.
@@ -71,12 +80,20 @@ func NewSchemaCommand() *cobra.Command {
Short: "渐进查看命令 Schema (产品 / 分组 / 工具参数)",
Long: `查看当前可运行命令的 Schema 元数据。
不带参数时列出产品和工具数量;传产品或分组路径逐层展开;传具体工具路径输出扁平参数 Schema(对齐 GWS:parameters 内联 required,键为 CLI flag)。普通 Agent 查询应使用 --compact:它按稳定字段白名单输出选参、约束、安全语义和已评审的返回契约。省略 --compact 的 full leaf 保留参数映射、接口绑定和 provenance,仅用于定向审计;--all 输出全部工具的完整 leaf Schema,用于审计/CI。helper、MCP 与本地 Cobra 命令均须通过 ContractFinal.Identity 声明进入收集的身份集,并从同一声明装配的 ToolSpec 投影;查询不执行服务发现或临时合成第二份 Schema。`,
Args: cobra.MaximumNArgs(1),
不带参数时列出产品和工具数量;传产品或分组路径逐层展开;传具体工具路径输出扁平参数 Schema(对齐 GWS:parameters 内联 required,键为 CLI flag)。普通 Agent 查询应使用 --compact:它按稳定字段白名单输出选参、约束、安全语义和已评审的返回契约。省略 --compact 的 full leaf 保留参数映射、接口绑定和 provenance,仅用于定向审计;--all 输出全部工具的完整 leaf Schema,用于审计/CI。helper、MCP 与本地 Cobra 命令均须通过 ContractFinal.Identity 声明进入收集的身份集,并从同一声明装配的 ToolSpec 投影;查询不执行服务发现或临时合成第二份 Schema。
路径形态:多个位置参数会按空格拼接并作为 CLI path 解析,等价于加引号的形式和 --cli-path,例如 dws schema chat message read-status 与 dws schema "chat message read-status" 相同。canonical path 含点号,必须作为单个参数整体传入(dws schema chat.query_msg_read_status);把 canonical 拆成多个 token 会被当作 CLI path 而解析失败。`,
Args: cobra.MaximumNArgs(16),
DisableAutoGenTag: true,
RunE: func(cmd *cobra.Command, args []string) error {
all, _ := cmd.Flags().GetBool("all")
compact, _ := cmd.Flags().GetBool("compact")
expectedSourceHash, _ := cmd.Flags().GetString("expected-source-hash")
expectedSurfaceHash, _ := cmd.Flags().GetString("expected-surface-hash")
if (expectedSourceHash == "") != (expectedSurfaceHash == "") {
return apperrors.NewValidation("--expected-source-hash and --expected-surface-hash must be provided together", apperrors.WithReason("incomplete_catalog_version"))
}
versionedInspect := expectedSourceHash != ""
cliPath, _ := cmd.Flags().GetString("cli-path")
cliPath = strings.TrimSpace(cliPath)
if cliPath != "" && len(args) > 0 {
@@ -90,10 +107,34 @@ func NewSchemaCommand() *cobra.Command {
}
if cliPath != "" {
args = []string{cliPath}
} else if len(args) > 1 {
// Preserve the quoted path form while accepting the equivalent
// natural tokenized form: dws schema chat message.
args = []string{strings.Join(args, " ")}
}
if versionedInspect {
if all || len(args) == 0 || !compact {
return apperrors.NewValidation("versioned Schema inspect requires one tool path and --compact", apperrors.WithReason("invalid_inspect_request"))
}
for _, name := range []string{"fields", "jq"} {
if flag := cmd.Flag(name); flag != nil && flag.Changed {
return apperrors.NewValidation("schema-inspect.v1 does not support --"+name, apperrors.WithReason("unsupported_output_filter"))
}
}
if flag := cmd.Flag("format"); flag != nil && flag.Changed && strings.TrimSpace(flag.Value.String()) != "json" {
return apperrors.NewValidation("schema-inspect.v1 only supports --format json", apperrors.WithReason("unsupported_output_format"))
}
}
if err := schemaCommandCatalogError(); err != nil {
return fmt.Errorf("load typed Schema registry: %w", err)
}
loaded := deliverySchemaCatalog()
if expectedSourceHash != "" && expectedSourceHash != loaded.Snapshot.SourceHash {
return apperrors.NewDiscovery("Catalog changed before Schema inspect", apperrors.WithReason("catalog_changed"))
}
if expectedSurfaceHash != "" && expectedSurfaceHash != loaded.Snapshot.SurfaceHash {
return apperrors.NewDiscovery("Catalog surface changed before Schema inspect", apperrors.WithReason("catalog_changed"))
}
var payload map[string]any
var err error
if all {
@@ -109,12 +150,22 @@ func NewSchemaCommand() *cobra.Command {
if compact {
payload = stripSchemaPayloadCompact(payload)
}
if versionedInspect {
return output.WriteFiltered(cmd.OutOrStdout(), output.FormatJSON, SchemaInspectV1Response{
Version: "schema-inspect.v1",
Catalog: CatalogVersionRef{SourceHash: loaded.Snapshot.SourceHash, SurfaceHash: loaded.Snapshot.SurfaceHash},
ToolSpec: payload,
}, "", "")
}
return output.WriteFiltered(cmd.OutOrStdout(), output.ResolveFormat(cmd, output.FormatJSON), payload, output.ResolveFields(cmd), output.ResolveJQ(cmd))
},
}
cmd.Flags().Bool("all", false, "输出全部工具的完整 leaf Schema(包括参数和约束,用于审计/CI)")
cmd.Flags().Bool("compact", false, "按稳定字段白名单输出 Agent 选参、约束、安全语义和返回契约")
cmd.Flags().String("cli-path", "", "按 CLI 命令路径查询")
cmd.Flags().String("expected-source-hash", "", "要求 Inspect 使用指定 Catalog source hash")
cmd.Flags().String("expected-surface-hash", "", "要求 Inspect 使用指定 Catalog surface hash")
cmd.AddCommand(newSchemaSearchCommand())
return cmd
}
+6
View File
@@ -35,6 +35,12 @@ func resetSchemaSourceRootAtomicForTest() {
schemaSourceRoot = atomic.Value{}
}
// resetDeliveryToolSearchEngineStateForTest drops the memoized default Tool
// Search engine so a test can rebuild it against a freshly installed Catalog.
func resetDeliveryToolSearchEngineStateForTest() {
resetDeliveryToolSearchEngineState()
}
// clearDeclaredDryRunCapabilitiesForTest resets the declared index (tests only).
func clearDeclaredDryRunCapabilitiesForTest() {
declaredDryRunCapabilities.Range(func(key, _ any) bool {
+4 -1
View File
@@ -325,7 +325,10 @@ func schemaPayloadFromLoadedCatalog(loaded loadedSchemaCatalog, args []string) (
}
}
}
return nil, apperrors.NewValidation("unknown runtime schema path " + strconvQuote(raw))
// The message carries the path-form boundary because it is the one surface
// an Agent is guaranteed to see at failure time.
return nil, apperrors.NewValidation("unknown runtime schema path " + strconvQuote(raw) +
": canonical paths contain dots and must be a single argument (e.g. dws schema chat.query_msg_read_status), while space-separated tokens resolve as a CLI path")
}
func schemaCatalogSnapshotHash(snapshot SchemaCatalogSnapshot) string {
+1 -1
View File
@@ -51,7 +51,7 @@ var reviewedRuntimeSchemaExclusionGroups = []runtimeSchemaExclusionGroup{
"profile list",
"profile switch",
"profile use",
"schema",
"schema search",
"skill get",
"skill install",
"skill search",
+18 -2
View File
@@ -1127,21 +1127,37 @@ func putRawJSON(payload map[string]any, key string, raw json.RawMessage) error {
return nil
}
// rawJSONValue decodes a JSON value that may come from an untrusted source, so
// it validates before decoding. Callers holding output that json.Marshal just
// produced should use typedJSONValue instead of paying the validation scan.
func rawJSONValue(raw json.RawMessage) (any, error) {
if !json.Valid(raw) {
return nil, fmt.Errorf("invalid JSON value")
}
return decodeValidJSONValue(raw), nil
}
// decodeValidJSONValue decodes JSON whose validity the caller has already
// established, either by json.Valid or by having just marshaled it. Decode
// errors are unreachable under that precondition and are therefore discarded,
// exactly as this path behaved when the decode was inlined into rawJSONValue.
func decodeValidJSONValue(raw json.RawMessage) any {
decoder := json.NewDecoder(bytes.NewReader(raw))
decoder.UseNumber()
var value any
_ = decoder.Decode(&value)
return value, nil
return value
}
// typedJSONValue projects a typed value into the generic JSON shape the payload
// renderers consume. json.Marshal output is valid by construction, so this path
// decodes it directly: routing through rawJSONValue re-scanned every marshaled
// document with json.Valid, which measured ~34% of Schema Catalog assembly time
// across the 1121-tool set (26.0s -> 17.2s for the internal/app schema suite).
func typedJSONValue(value any) (any, error) {
data, err := json.Marshal(value)
if err != nil {
return nil, err
}
return rawJSONValue(data)
return decodeValidJSONValue(data), nil
}
+203
View File
@@ -0,0 +1,203 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cli
import (
"bytes"
"encoding/json"
"fmt"
"io"
"strings"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
"github.com/spf13/cobra"
)
// ToolSearchV1Request is the stable stdin transport used by Agent hosts.
type ToolSearchV1Request struct {
Version string `json:"version"`
Query string `json:"query"`
Subqueries []string `json:"subqueries,omitempty"`
Limit int `json:"limit,omitempty"`
CandidateLimit int `json:"candidate_limit,omitempty"`
ProductIDs []string `json:"product_ids,omitempty"`
Effects []string `json:"effects,omitempty"`
ExcludeCanonical []string `json:"exclude_canonical,omitempty"`
}
var schemaSearchNewEngine = NewDeliveryToolSearchEngineWithConfig
// schemaSearchEngineConfig resolves the search config the same way
// NewDeliveryToolSearchEngine does (defaults + DWS_TOOL_SEARCH_* env
// overrides), then lets an explicit --explain flag win over the env value.
func schemaSearchEngineConfig(cmd *cobra.Command) ToolSearchConfig {
config := ResolveToolSearchConfigFromEnv(DefaultToolSearchConfig())
if explain, _ := cmd.Flags().GetBool("explain"); explain {
config.Explain = true
}
return config
}
func newSchemaSearchCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "search",
Short: "按自然语言搜索可用命令 Schema",
Long: `按自然语言或精确身份检索 ToolReference。
输出固定为完整、紧凑的 tool-search.v1 JSON envelope。为保证版本、Catalog
绑定和响应字节预算可验证,本命令不支持 --fields、--jq 或非 JSON --format。
DWS 对 Agent 是一个元工具:已知子命令直接执行,未知子命令才使用 search。
选中候选后,必须使用 response catalog 的 source/surface hash Inspect canonical_path,
再按 primary_cli_path 执行;Search 结果本身不授权、不执行。`,
Example: ` dws schema search --query "查询群消息已读状态"
dws schema chat.query_msg_read_status --compact --format json \
--expected-source-hash "<search.catalog.source_hash>" \
--expected-surface-hash "<search.catalog.surface_hash>"
printf '%s' '{"version":"tool-search.v1","query":"发群文件并确认已读","subqueries":["给群里发送文件消息","查询群消息已读状态"]}' |
dws schema search --request-json -`,
Args: cobra.NoArgs,
DisableAutoGenTag: true,
RunE: func(cmd *cobra.Command, _ []string) error {
if err := validateSchemaSearchOutputFlags(cmd); err != nil {
return err
}
request, err := schemaSearchRequest(cmd)
if err != nil {
return err
}
engine, err := schemaSearchNewEngine(schemaSearchEngineConfig(cmd))
if err != nil {
return err
}
internal := ToolSearchRequest{
Query: request.Query,
Limit: request.Limit,
CandidateLimit: request.CandidateLimit,
ProductIDs: request.ProductIDs,
Effects: request.Effects,
ExcludeCanonicalPaths: request.ExcludeCanonical,
}
var response ToolSearchResponse
if len(request.Subqueries) > 0 {
response, err = engine.SearchSubqueries(cmd.Context(), request.Subqueries, internal)
} else {
response, err = engine.Search(cmd.Context(), internal)
}
if err != nil {
return err
}
// The response budget is defined over this compact wire encoding. Do
// not run it through pretty/table formatting or post-filtering because
// Agent hosts require one complete versioned envelope.
if err := json.NewEncoder(cmd.OutOrStdout()).Encode(response); err != nil {
return fmt.Errorf("encode tool-search.v1 response: %w", err)
}
return nil
},
}
cmd.Flags().String("query", "", "自然语言意图或 canonical/CLI path")
cmd.Flags().Int("limit", defaultToolSearchLimit, fmt.Sprintf("返回引用数量(1-%d;0 表示沿用默认值 %d)", maxToolSearchLimit, defaultToolSearchLimit))
cmd.Flags().Int("candidate-limit", defaultToolSearchCandidateLimit, fmt.Sprintf("每路召回候选数量(1-%d;0 表示沿用默认值 %d)", maxToolSearchCandidateLimit, defaultToolSearchCandidateLimit))
cmd.Flags().StringSlice("product", nil, "限制产品 ID,可重复或逗号分隔")
cmd.Flags().StringSlice("effect", nil, "限制 effect,可重复或逗号分隔")
cmd.Flags().StringSlice("exclude", nil, "排除 canonical path,可重复或逗号分隔")
cmd.Flags().String("request-json", "", `从 stdin 读取 JSON;值必须为 -;必含 version="tool-search.v1",动作分解字段为 subqueries`)
cmd.Flags().Bool("explain", false, "开发诊断:输出 score_breakdown 含原始排序分(公开默认响应不含分数;仅用于离线调参,不建议线上 Agent 使用;会增大响应体积,建议配合 --limit 3)。score_breakdown.score 是未量化的原始分,排序键使用量化到 1e-9 的值,两者末位可能不同,不要用它复算候选顺序")
return cmd
}
func validateSchemaSearchOutputFlags(cmd *cobra.Command) error {
for _, name := range []string{"fields", "jq"} {
if flag := cmd.Flag(name); flag != nil && flag.Changed {
return apperrors.NewValidation(
"schema search returns one complete tool-search.v1 envelope and does not support --"+name,
apperrors.WithReason("unsupported_output_filter"),
)
}
}
if flag := cmd.Flag("format"); flag != nil && flag.Changed && strings.TrimSpace(flag.Value.String()) != "json" {
return apperrors.NewValidation(
"schema search only supports --format json",
apperrors.WithReason("unsupported_output_format"),
)
}
return nil
}
func schemaSearchRequest(cmd *cobra.Command) (ToolSearchV1Request, error) {
requestJSON, _ := cmd.Flags().GetString("request-json")
requestJSON = strings.TrimSpace(requestJSON)
if requestJSON != "" {
if requestJSON != "-" {
return ToolSearchV1Request{}, apperrors.NewValidation("--request-json only accepts - (stdin)", apperrors.WithReason("invalid_transport"))
}
for _, name := range []string{"query", "limit", "candidate-limit", "product", "effect", "exclude"} {
if cmd.Flags().Changed(name) {
return ToolSearchV1Request{}, apperrors.NewValidation("--request-json cannot be combined with --"+name, apperrors.WithReason("ambiguous_input"))
}
}
request, err := decodeToolSearchV1Request(cmd.InOrStdin())
if err != nil {
return ToolSearchV1Request{}, err
}
return validateToolSearchV1Request(request)
}
query, _ := cmd.Flags().GetString("query")
limit, _ := cmd.Flags().GetInt("limit")
candidateLimit, _ := cmd.Flags().GetInt("candidate-limit")
products, _ := cmd.Flags().GetStringSlice("product")
effects, _ := cmd.Flags().GetStringSlice("effect")
excluded, _ := cmd.Flags().GetStringSlice("exclude")
return validateToolSearchV1Request(ToolSearchV1Request{
Version: "tool-search.v1",
Query: query,
Limit: limit,
CandidateLimit: candidateLimit,
ProductIDs: products,
Effects: effects,
ExcludeCanonical: excluded,
})
}
func decodeToolSearchV1Request(reader io.Reader) (ToolSearchV1Request, error) {
payload, err := io.ReadAll(io.LimitReader(reader, maxToolSearchRequestBytes+1))
if err != nil {
return ToolSearchV1Request{}, apperrors.NewValidation("read tool search request: "+err.Error(), apperrors.WithReason("read_request_failed"))
}
if len(payload) > maxToolSearchRequestBytes {
return ToolSearchV1Request{}, apperrors.NewValidation(fmt.Sprintf("tool search request exceeds %d bytes", maxToolSearchRequestBytes), apperrors.WithReason("request_too_large"))
}
decoder := json.NewDecoder(bytes.NewReader(payload))
decoder.DisallowUnknownFields()
var request ToolSearchV1Request
if err := decoder.Decode(&request); err != nil {
return ToolSearchV1Request{}, apperrors.NewValidation("decode tool-search.v1 request: "+err.Error(), apperrors.WithReason("invalid_request_json"))
}
var trailing any
if err := decoder.Decode(&trailing); err != io.EOF {
if err == nil {
return ToolSearchV1Request{}, apperrors.NewValidation("tool search request must contain exactly one JSON object", apperrors.WithReason("invalid_request_json"))
}
return ToolSearchV1Request{}, apperrors.NewValidation("decode trailing tool search input: "+err.Error(), apperrors.WithReason("invalid_request_json"))
}
return request, nil
}
func validateToolSearchV1Request(request ToolSearchV1Request) (ToolSearchV1Request, error) {
if request.Version != "tool-search.v1" {
return ToolSearchV1Request{}, apperrors.NewValidation(`tool search request must include "version":"tool-search.v1"; action decomposition uses the "subqueries" array`, apperrors.WithReason("unsupported_version"))
}
return request, nil
}
@@ -0,0 +1,95 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
package cli
import (
"bytes"
"encoding/json"
"errors"
"strings"
"testing"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
"github.com/spf13/cobra"
)
func TestSchemaSearchCommandExactAndVersioned(t *testing.T) {
installSchemaSearchTestEngine(t)
root := &cobra.Command{Use: "dws"}
root.AddCommand(NewSchemaCommand())
var stdout bytes.Buffer
root.SetOut(&stdout)
root.SetArgs([]string{"schema", "search", "--query", "chat.read_status"})
if err := root.Execute(); err != nil {
t.Fatalf("schema search execute: %v", err)
}
var response ToolSearchResponse
if err := json.Unmarshal(stdout.Bytes(), &response); err != nil {
t.Fatalf("decode response: %v\n%s", err, stdout.String())
}
if response.Version != "tool-search.v1" || response.Strategy != "exact_guard" {
t.Fatalf("response = %#v", response)
}
if len(response.Candidates) != 1 || response.Candidates[0].CanonicalPath != "chat.read_status" {
t.Fatalf("candidates = %#v", response.Candidates)
}
wantCatalog := CatalogVersionRef{SourceHash: "source-test", SurfaceHash: "surface-test"}
if response.Catalog != wantCatalog {
t.Fatalf("Catalog = %#v, want %#v", response.Catalog, wantCatalog)
}
for _, removed := range []string{"external_ranking", "degraded", "degraded_reason_code", "warning_codes", "_fallback"} {
if strings.Contains(stdout.String(), removed) {
t.Fatalf("removed fallback field %q leaked into response: %s", removed, stdout.String())
}
}
}
func TestSchemaSearchRequestRejectsUnboundedOrAmbiguousInput(t *testing.T) {
if _, err := decodeToolSearchV1Request(strings.NewReader(`{"version":"tool-search.v1","query":"x","unknown":true}`)); validationReason(err) != "invalid_request_json" {
t.Fatal("unknown field was accepted")
}
if _, err := decodeToolSearchV1Request(strings.NewReader(`{"version":"tool-search.v1","query":"x","external_ranking":{}}`)); validationReason(err) != "invalid_request_json" {
t.Fatal("removed external_ranking field was accepted")
}
oversized := strings.NewReader(strings.Repeat("x", maxToolSearchRequestBytes+1))
if _, err := decodeToolSearchV1Request(oversized); validationReason(err) != "request_too_large" {
t.Fatal("oversized request was accepted")
}
_, err := validateToolSearchV1Request(ToolSearchV1Request{Query: "x", Subqueries: []string{"x"}})
if validationReason(err) != "unsupported_version" || !strings.Contains(err.Error(), `"version":"tool-search.v1"`) || !strings.Contains(err.Error(), "subqueries") {
t.Fatalf("missing-version guidance = %v", err)
}
}
func TestSchemaSearchHelpDocumentsStructuredRequest(t *testing.T) {
command := newSchemaSearchCommand()
var output bytes.Buffer
command.SetOut(&output)
command.SetArgs([]string{"--help"})
if err := command.Execute(); err != nil {
t.Fatalf("schema search help: %v", err)
}
for _, required := range []string{`"version":"tool-search.v1"`, `"subqueries"`, "--request-json -", "不支持 --fields、--jq", "非 JSON --format"} {
if !strings.Contains(output.String(), required) {
t.Fatalf("schema search help omits %q:\n%s", required, output.String())
}
}
}
func installSchemaSearchTestEngine(t *testing.T) {
t.Helper()
testseam.Swap(t, &schemaSearchNewEngine, func(ToolSearchConfig) (*ToolSearchEngine, error) {
return newToolSearchTestEngine(t), nil
})
}
func validationReason(err error) string {
var typed *apperrors.Error
if errors.As(err, &typed) && typed.Category == apperrors.CategoryValidation {
return typed.Reason
}
return ""
}
+4 -1
View File
@@ -63,12 +63,15 @@ func storeSchemaSourceRootFn(fn func() *cobra.Command) {
}
// resetDeliverySchemaCatalogState clears the lazy Catalog delivery Once/caches
// so the next deliverySchemaCatalog() reassembles.
// so the next deliverySchemaCatalog() reassembles. The memoized Tool Search
// engine is a Catalog derivative (it indexes one generation and caches its
// source/surface hashes), so it is invalidated here as well.
func resetDeliverySchemaCatalogState() {
runtimeDeliverySchemaCatalogOnce = sync.Once{}
runtimeDeliverySchemaCatalog = loadedSchemaCatalog{}
runtimeDeliverySchemaCatalogErr = nil
runtimeDeliverySchemaCatalogLazyCount.Store(0)
resetDeliveryToolSearchEngineState()
}
// resetSchemaDeliveryState clears ResolveMeta lookup state and Catalog delivery
+19
View File
@@ -7,11 +7,30 @@ package cli
import (
"bytes"
"encoding/json"
"strings"
"testing"
"github.com/spf13/cobra"
)
func TestSchemaHelpDocumentsCanonicalAndCLIPathForms(t *testing.T) {
// Positional args are joined with spaces and resolved as a CLI path, so the
// tokenized form is not a canonical-path spelling. Help must keep saying so:
// an Agent that splits a canonical path on the dot otherwise gets an
// "unknown runtime schema path" error with no explanation of the boundary.
long := NewSchemaCommand().Long
for _, required := range []string{
"CLI path",
"canonical path",
`dws schema "chat message read-status"`,
"dws schema chat.query_msg_read_status",
} {
if !strings.Contains(long, required) {
t.Fatalf("schema long help omits the path-form boundary %q:\n%s", required, long)
}
}
}
func TestSchemaUsesDeliveryCatalogWithoutRuntimeLoad(t *testing.T) {
root := &cobra.Command{Use: "dws"}
root.AddCommand(NewSchemaCommand())
File diff suppressed because it is too large Load Diff
+490
View File
@@ -0,0 +1,490 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cli
import (
"fmt"
"math/rand"
"sort"
"strings"
)
const (
ToolSearchAgentArmDirectSchema = "direct_schema"
ToolSearchAgentArmSearchInspect = "search_inspect"
)
// Bootstrap resampling uses fixed seeds so a report can be re-derived from the
// same paired runs. The seeds are published in the report so an auditor never
// has to read them out of the source to reproduce an interval. The recovery
// slice uses a distinct seed because it resamples a different pair subset.
const (
toolSearchAgentABBootstrapSamples = 10000
toolSearchAgentABBootstrapSeed = 927
toolSearchAgentABRecoveryBootstrapSeed = 928
)
// ToolSearchAgentABInput is populated by an Agent runner. Every case/trial
// must have both arms so the comparison remains paired.
type ToolSearchAgentABInput struct {
Version string `json:"version"`
Catalog CatalogVersionRef `json:"catalog"`
Runs []ToolSearchAgentABRun `json:"runs"`
}
// ToolSearchAgentABRun records end-to-end outcomes rather than retrieval
// proxies. ContextTokens must come from the selected model's tokenizer.
type ToolSearchAgentABRun struct {
CaseID string `json:"case_id"`
Trial int `json:"trial"`
Arm string `json:"arm"`
TaskCompleted bool `json:"task_completed"`
CorrectToolPlan bool `json:"correct_tool_plan"`
UnsafeAction bool `json:"unsafe_action"`
RecoveryAttempted bool `json:"recovery_attempted"`
RecoverySucceeded bool `json:"recovery_succeeded"`
ContextTokens float64 `json:"context_tokens"`
ToolCalls float64 `json:"tool_calls"`
LatencyMS float64 `json:"latency_ms"`
}
type ToolSearchAgentABArmMetrics struct {
Runs int `json:"runs"`
TaskSuccessRate float64 `json:"task_success_rate"`
CorrectPlanRate float64 `json:"correct_plan_rate"`
UnsafeActionRate float64 `json:"unsafe_action_rate"`
RecoverySuccessRate float64 `json:"recovery_success_rate"`
RecoveryAttempts int `json:"recovery_attempts"`
AverageContextTokens float64 `json:"average_context_tokens"`
AverageToolCalls float64 `json:"average_tool_calls"`
AverageLatencyMS float64 `json:"average_latency_ms"`
}
// ToolSearchAgentABDelta is search_inspect minus direct_schema. A positive
// value is desirable for success/plan/recovery and undesirable for unsafe
// actions, context, calls, and latency.
type ToolSearchAgentABDelta struct {
DirectSchema float64 `json:"direct_schema"`
SearchInspect float64 `json:"search_inspect"`
Delta float64 `json:"delta"`
CI95Low float64 `json:"ci95_low"`
CI95High float64 `json:"ci95_high"`
}
type ToolSearchAgentABReport struct {
Version string `json:"version"`
Catalog CatalogVersionRef `json:"catalog"`
Cases int `json:"cases"`
PairedTrials int `json:"paired_trials"`
DirectSchema ToolSearchAgentABArmMetrics `json:"direct_schema"`
SearchInspect ToolSearchAgentABArmMetrics `json:"search_inspect"`
Deltas map[string]ToolSearchAgentABDelta `json:"deltas"`
// BootstrapSamples/BootstrapSeed/RecoveryBootstrapSeed are published so an
// auditor can reproduce every interval without reading the source.
BootstrapSamples int `json:"bootstrap_samples"`
BootstrapSeed int64 `json:"bootstrap_seed"`
RecoveryBootstrapSeed int64 `json:"recovery_bootstrap_seed"`
Method string `json:"method"`
}
// ToolSearchAgentPlanOutput is the strict final response of a non-executing
// Agent planning run. Gold tools remain in the local workflow fixture and are
// never sent to the evaluated model.
type ToolSearchAgentPlanOutput struct {
Results []ToolSearchAgentPlanResult `json:"results"`
}
type ToolSearchAgentPlanResult struct {
ID string `json:"id"`
CanonicalPaths []string `json:"canonical_paths"`
}
type ToolSearchAgentPlanMetrics struct {
Cases int `json:"cases"`
CompleteRate float64 `json:"complete_rate"`
ExactMinimalRate float64 `json:"exact_minimal_rate"`
RequiredToolRecall float64 `json:"required_tool_recall"`
PlanPrecision float64 `json:"plan_precision"`
RequiredTools int `json:"required_tools"`
PredictedTools int `json:"predicted_tools"`
UnnecessaryTools int `json:"unnecessary_tools"`
}
type ToolSearchAgentPlanningABReport struct {
Version string `json:"version"`
Model string `json:"model"`
DirectSchema ToolSearchAgentPlanMetrics `json:"direct_schema"`
SearchInspect ToolSearchAgentPlanMetrics `json:"search_inspect"`
Delta map[string]float64 `json:"delta_search_minus_direct"`
Limitations []string `json:"limitations"`
}
// ScoreToolSearchAgentPlanningAB scores an answer-free paired planning smoke
// run against reviewed workflow requirements. It measures plan coverage and
// minimality only; it does not claim that any business task was executed.
func ScoreToolSearchAgentPlanningAB(model string, workflows []ToolSearchWorkflowEvaluationCase, direct, search ToolSearchAgentPlanOutput) (ToolSearchAgentPlanningABReport, error) {
if err := deliverySchemaCatalogError(); err != nil {
return ToolSearchAgentPlanningABReport{}, fmt.Errorf("assemble typed Schema registry for Agent planning score: %w", err)
}
return scoreToolSearchAgentPlanningAB(deliverySchemaCatalog().Index, model, workflows, direct, search)
}
func scoreToolSearchAgentPlanningAB(index SchemaIndex, model string, workflows []ToolSearchWorkflowEvaluationCase, direct, search ToolSearchAgentPlanOutput) (ToolSearchAgentPlanningABReport, error) {
if strings.TrimSpace(model) == "" {
return ToolSearchAgentPlanningABReport{}, fmt.Errorf("Agent planning A/B requires a model identifier")
}
directMetrics, err := scoreToolSearchAgentPlans(index, workflows, direct)
if err != nil {
return ToolSearchAgentPlanningABReport{}, fmt.Errorf("score direct_schema plans: %w", err)
}
searchMetrics, err := scoreToolSearchAgentPlans(index, workflows, search)
if err != nil {
return ToolSearchAgentPlanningABReport{}, fmt.Errorf("score search_inspect plans: %w", err)
}
return ToolSearchAgentPlanningABReport{
Version: "tool-search-agent-planning-ab.v1",
Model: strings.TrimSpace(model),
DirectSchema: directMetrics,
SearchInspect: searchMetrics,
Delta: map[string]float64{
"complete_rate": searchMetrics.CompleteRate - directMetrics.CompleteRate,
"exact_minimal_rate": searchMetrics.ExactMinimalRate - directMetrics.ExactMinimalRate,
"required_tool_recall": searchMetrics.RequiredToolRecall - directMetrics.RequiredToolRecall,
"plan_precision": searchMetrics.PlanPrecision - directMetrics.PlanPrecision,
"unnecessary_tools": float64(searchMetrics.UnnecessaryTools - directMetrics.UnnecessaryTools),
},
Limitations: []string{
"one batch and one model trial per arm; no confidence interval",
"reviewed ten-workflow diagnostic fixture is not an independent release test",
"planning only: no parameters, business execution, recovery, latency, or tokenizer usage is scored",
"extra prerequisites can be operationally valid even when absent from the minimal reviewed gold",
},
}, nil
}
func scoreToolSearchAgentPlans(index SchemaIndex, workflows []ToolSearchWorkflowEvaluationCase, output ToolSearchAgentPlanOutput) (ToolSearchAgentPlanMetrics, error) {
byID := make(map[string]ToolSearchAgentPlanResult, len(output.Results))
for _, result := range output.Results {
if strings.TrimSpace(result.ID) == "" || len(result.CanonicalPaths) == 0 {
return ToolSearchAgentPlanMetrics{}, fmt.Errorf("plan result requires id and at least one canonical path")
}
if _, exists := byID[result.ID]; exists {
return ToolSearchAgentPlanMetrics{}, fmt.Errorf("duplicate plan result %q", result.ID)
}
seen := make(map[string]bool, len(result.CanonicalPaths))
for _, canonical := range result.CanonicalPaths {
if _, ok := index.Resolve(canonical); !ok {
return ToolSearchAgentPlanMetrics{}, fmt.Errorf("plan result %q contains unknown canonical %q", result.ID, canonical)
}
if seen[canonical] {
return ToolSearchAgentPlanMetrics{}, fmt.Errorf("plan result %q repeats canonical %q", result.ID, canonical)
}
seen[canonical] = true
}
byID[result.ID] = result
}
metrics := ToolSearchAgentPlanMetrics{Cases: len(workflows)}
var completeCases, exactCases, hits int
for _, workflow := range workflows {
result, ok := byID[workflow.ID]
if !ok {
return ToolSearchAgentPlanMetrics{}, fmt.Errorf("missing plan result %q", workflow.ID)
}
delete(byID, workflow.ID)
gold := stringSet(workflow.Required)
caseHits := 0
for _, canonical := range result.CanonicalPaths {
if gold[canonical] {
caseHits++
}
}
hits += caseHits
metrics.RequiredTools += len(workflow.Required)
metrics.PredictedTools += len(result.CanonicalPaths)
if caseHits == len(workflow.Required) {
completeCases++
}
if stringSlicesEqual(result.CanonicalPaths, workflow.Required) {
exactCases++
}
}
if len(byID) > 0 {
unexpected := make([]string, 0, len(byID))
for id := range byID {
unexpected = append(unexpected, id)
}
sort.Strings(unexpected)
return ToolSearchAgentPlanMetrics{}, fmt.Errorf("unexpected plan results: %s", strings.Join(unexpected, ", "))
}
if metrics.Cases > 0 {
metrics.CompleteRate = float64(completeCases) / float64(metrics.Cases)
metrics.ExactMinimalRate = float64(exactCases) / float64(metrics.Cases)
}
if metrics.RequiredTools > 0 {
metrics.RequiredToolRecall = float64(hits) / float64(metrics.RequiredTools)
}
if metrics.PredictedTools > 0 {
metrics.PlanPrecision = float64(hits) / float64(metrics.PredictedTools)
}
metrics.UnnecessaryTools = metrics.PredictedTools - hits
return metrics, nil
}
func stringSlicesEqual(left, right []string) bool {
if len(left) != len(right) {
return false
}
for index := range left {
if left[index] != right[index] {
return false
}
}
return true
}
type toolSearchAgentPair struct {
caseID string
direct ToolSearchAgentABRun
search ToolSearchAgentABRun
}
// AggregateToolSearchAgentAB validates a paired experiment and computes
// case-cluster bootstrap confidence intervals with a fixed seed. Trials of the
// same task are averaged inside a case before resampling so repeated model
// seeds are not treated as independent tasks.
func AggregateToolSearchAgentAB(input ToolSearchAgentABInput) (ToolSearchAgentABReport, error) {
if input.Version != "tool-search-agent-ab.v1" {
return ToolSearchAgentABReport{}, fmt.Errorf("unsupported Agent A/B input version %q", input.Version)
}
if strings.TrimSpace(input.Catalog.SourceHash) == "" || strings.TrimSpace(input.Catalog.SurfaceHash) == "" {
return ToolSearchAgentABReport{}, fmt.Errorf("Agent A/B input requires both Catalog hashes")
}
type key struct {
caseID string
trial int
}
paired := make(map[key]*toolSearchAgentPair)
for _, run := range input.Runs {
if strings.TrimSpace(run.CaseID) == "" || run.Trial < 0 {
return ToolSearchAgentABReport{}, fmt.Errorf("Agent A/B run requires case_id and non-negative trial")
}
if run.ContextTokens < 0 || run.ToolCalls < 0 || run.LatencyMS < 0 {
return ToolSearchAgentABReport{}, fmt.Errorf("Agent A/B run %s/%d has a negative cost", run.CaseID, run.Trial)
}
if run.RecoverySucceeded && !run.RecoveryAttempted {
return ToolSearchAgentABReport{}, fmt.Errorf("Agent A/B run %s/%d marks recovery success without an attempt", run.CaseID, run.Trial)
}
itemKey := key{caseID: run.CaseID, trial: run.Trial}
pair := paired[itemKey]
if pair == nil {
pair = &toolSearchAgentPair{caseID: run.CaseID}
paired[itemKey] = pair
}
switch run.Arm {
case ToolSearchAgentArmDirectSchema:
if pair.direct.Arm != "" {
return ToolSearchAgentABReport{}, fmt.Errorf("duplicate direct_schema run for %s/%d", run.CaseID, run.Trial)
}
pair.direct = run
case ToolSearchAgentArmSearchInspect:
if pair.search.Arm != "" {
return ToolSearchAgentABReport{}, fmt.Errorf("duplicate search_inspect run for %s/%d", run.CaseID, run.Trial)
}
pair.search = run
default:
return ToolSearchAgentABReport{}, fmt.Errorf("unknown Agent A/B arm %q", run.Arm)
}
}
if len(paired) == 0 {
return ToolSearchAgentABReport{}, fmt.Errorf("Agent A/B input contains no runs")
}
pairs := make([]toolSearchAgentPair, 0, len(paired))
caseSet := make(map[string]bool)
for itemKey, pair := range paired {
if pair.direct.Arm == "" || pair.search.Arm == "" {
return ToolSearchAgentABReport{}, fmt.Errorf("Agent A/B case %s/%d is not paired", itemKey.caseID, itemKey.trial)
}
pairs = append(pairs, *pair)
caseSet[itemKey.caseID] = true
}
sort.Slice(pairs, func(i, j int) bool {
if pairs[i].caseID == pairs[j].caseID {
return pairs[i].direct.Trial < pairs[j].direct.Trial
}
return pairs[i].caseID < pairs[j].caseID
})
directMetrics := aggregateToolSearchAgentArm(pairs, false)
searchMetrics := aggregateToolSearchAgentArm(pairs, true)
metrics := []struct {
name string
value func(ToolSearchAgentABRun) float64
direct float64
search float64
}{
{"task_success_rate", func(run ToolSearchAgentABRun) float64 { return boolFloat(run.TaskCompleted) }, directMetrics.TaskSuccessRate, searchMetrics.TaskSuccessRate},
{"correct_plan_rate", func(run ToolSearchAgentABRun) float64 { return boolFloat(run.CorrectToolPlan) }, directMetrics.CorrectPlanRate, searchMetrics.CorrectPlanRate},
{"unsafe_action_rate", func(run ToolSearchAgentABRun) float64 { return boolFloat(run.UnsafeAction) }, directMetrics.UnsafeActionRate, searchMetrics.UnsafeActionRate},
{"context_tokens", func(run ToolSearchAgentABRun) float64 { return run.ContextTokens }, directMetrics.AverageContextTokens, searchMetrics.AverageContextTokens},
{"tool_calls", func(run ToolSearchAgentABRun) float64 { return run.ToolCalls }, directMetrics.AverageToolCalls, searchMetrics.AverageToolCalls},
{"latency_ms", func(run ToolSearchAgentABRun) float64 { return run.LatencyMS }, directMetrics.AverageLatencyMS, searchMetrics.AverageLatencyMS},
}
deltas := make(map[string]ToolSearchAgentABDelta, len(metrics)+1)
for _, metric := range metrics {
low, high := toolSearchCaseClusterBootstrap(pairs, metric.value, toolSearchAgentABBootstrapSamples, toolSearchAgentABBootstrapSeed)
deltas[metric.name] = ToolSearchAgentABDelta{
DirectSchema: metric.direct,
SearchInspect: metric.search,
Delta: metric.search - metric.direct,
CI95Low: low,
CI95High: high,
}
}
if directMetrics.RecoveryAttempts > 0 && searchMetrics.RecoveryAttempts > 0 {
// Recovery is conditionally defined over attempted recoveries, so its
// interval uses the same paired tasks but maps non-attempts to zero only
// after both arms attempted recovery for that trial.
recoveryPairs := make([]toolSearchAgentPair, 0, len(pairs))
for _, pair := range pairs {
if pair.direct.RecoveryAttempted && pair.search.RecoveryAttempted {
recoveryPairs = append(recoveryPairs, pair)
}
}
if len(recoveryPairs) > 0 {
value := func(run ToolSearchAgentABRun) float64 { return boolFloat(run.RecoverySucceeded) }
low, high := toolSearchCaseClusterBootstrap(recoveryPairs, value, toolSearchAgentABBootstrapSamples, toolSearchAgentABRecoveryBootstrapSeed)
directRecovery := meanAgentRunValue(recoveryPairs, false, value)
searchRecovery := meanAgentRunValue(recoveryPairs, true, value)
deltas["recovery_success_rate_paired_attempts"] = ToolSearchAgentABDelta{
DirectSchema: directRecovery,
SearchInspect: searchRecovery,
Delta: searchRecovery - directRecovery,
CI95Low: low,
CI95High: high,
}
}
}
return ToolSearchAgentABReport{
Version: "tool-search-agent-ab-report.v1",
Catalog: input.Catalog,
Cases: len(caseSet),
PairedTrials: len(pairs),
DirectSchema: directMetrics,
SearchInspect: searchMetrics,
Deltas: deltas,
BootstrapSamples: toolSearchAgentABBootstrapSamples,
BootstrapSeed: toolSearchAgentABBootstrapSeed,
RecoveryBootstrapSeed: toolSearchAgentABRecoveryBootstrapSeed,
Method: fmt.Sprintf(
"paired by case_id/trial; 95%% percentile bootstrap clustered by case_id; %d resamples; fixed seed %d (recovery slice seed %d)",
toolSearchAgentABBootstrapSamples,
toolSearchAgentABBootstrapSeed,
toolSearchAgentABRecoveryBootstrapSeed,
),
}, nil
}
func aggregateToolSearchAgentArm(pairs []toolSearchAgentPair, search bool) ToolSearchAgentABArmMetrics {
metrics := ToolSearchAgentABArmMetrics{Runs: len(pairs)}
for _, pair := range pairs {
run := pair.direct
if search {
run = pair.search
}
metrics.TaskSuccessRate += boolFloat(run.TaskCompleted)
metrics.CorrectPlanRate += boolFloat(run.CorrectToolPlan)
metrics.UnsafeActionRate += boolFloat(run.UnsafeAction)
metrics.AverageContextTokens += run.ContextTokens
metrics.AverageToolCalls += run.ToolCalls
metrics.AverageLatencyMS += run.LatencyMS
if run.RecoveryAttempted {
metrics.RecoveryAttempts++
metrics.RecoverySuccessRate += boolFloat(run.RecoverySucceeded)
}
}
if metrics.Runs > 0 {
denominator := float64(metrics.Runs)
metrics.TaskSuccessRate /= denominator
metrics.CorrectPlanRate /= denominator
metrics.UnsafeActionRate /= denominator
metrics.AverageContextTokens /= denominator
metrics.AverageToolCalls /= denominator
metrics.AverageLatencyMS /= denominator
}
if metrics.RecoveryAttempts > 0 {
metrics.RecoverySuccessRate /= float64(metrics.RecoveryAttempts)
}
return metrics
}
func toolSearchCaseClusterBootstrap(pairs []toolSearchAgentPair, value func(ToolSearchAgentABRun) float64, samples int, seed int64) (float64, float64) {
byCase := make(map[string][]float64)
for _, pair := range pairs {
byCase[pair.caseID] = append(byCase[pair.caseID], value(pair.search)-value(pair.direct))
}
caseIDs := make([]string, 0, len(byCase))
caseMeans := make(map[string]float64, len(byCase))
for caseID, values := range byCase {
caseIDs = append(caseIDs, caseID)
for _, value := range values {
caseMeans[caseID] += value / float64(len(values))
}
}
sort.Strings(caseIDs)
if len(caseIDs) == 1 {
value := caseMeans[caseIDs[0]]
return value, value
}
random := rand.New(rand.NewSource(seed))
distribution := make([]float64, samples)
for sample := range samples {
var total float64
for range caseIDs {
total += caseMeans[caseIDs[random.Intn(len(caseIDs))]]
}
distribution[sample] = total / float64(len(caseIDs))
}
sort.Float64s(distribution)
return percentile(distribution, 0.025), percentile(distribution, 0.975)
}
func percentile(sorted []float64, quantile float64) float64 {
if len(sorted) == 0 {
return 0
}
index := int(quantile * float64(len(sorted)-1))
return sorted[index]
}
func meanAgentRunValue(pairs []toolSearchAgentPair, search bool, value func(ToolSearchAgentABRun) float64) float64 {
if len(pairs) == 0 {
return 0
}
var total float64
for _, pair := range pairs {
run := pair.direct
if search {
run = pair.search
}
total += value(run)
}
return total / float64(len(pairs))
}
func boolFloat(value bool) float64 {
if value {
return 1
}
return 0
}
+587
View File
@@ -0,0 +1,587 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cli
import (
"context"
"encoding/json"
"fmt"
"math"
"sort"
"unicode"
"unicode/utf8"
)
// ToolSearchWorkflowEvaluationCase is a reviewed diagnostic workflow. It is
// intentionally separate from release qrels: the small in-repository fixture
// measures mechanics and cannot approve a ranking configuration.
type ToolSearchWorkflowEvaluationCase struct {
ID string `json:"id"`
Query string `json:"query"`
Subqueries []string `json:"subqueries"`
Required []string `json:"required"`
}
// ToolSearchRetrievalMetrics reports deterministic ranking metrics. Values are
// fractions in [0,1], not percentages.
type ToolSearchRetrievalMetrics struct {
Cases int `json:"cases"`
RecallAt1 float64 `json:"recall_at_1"`
RecallAt5 float64 `json:"recall_at_5"`
MeanReciprocalRankAt5 float64 `json:"mean_reciprocal_rank_at_5"`
ZeroResultRate float64 `json:"zero_result_rate"`
}
// ToolSearchWorkflowMetrics measures whether every required tool is present in
// the shared Top-5 budget and the average required-tool recall.
type ToolSearchWorkflowMetrics struct {
Cases int `json:"cases"`
CompleteAt5 float64 `json:"complete_at_5"`
RequiredRecallAt5 float64 `json:"required_recall_at_5"`
}
// ToolSearchIdentityTrustMetrics exercises the complete delivered identity
// set. Exact filtered checks deliberately exclude each canonical path and
// verify that search abstains instead of falling through to a fuzzy sibling.
type ToolSearchIdentityTrustMetrics struct {
CanonicalCases int `json:"canonical_cases"`
CanonicalPassRate float64 `json:"canonical_pass_rate"`
PrimaryCLICases int `json:"primary_cli_cases"`
PrimaryCLIPassRate float64 `json:"primary_cli_pass_rate"`
AliasCases int `json:"alias_cases"`
AliasPassRate float64 `json:"alias_pass_rate"`
NFKCCases int `json:"nfkc_cases"`
NFKCPassRate float64 `json:"nfkc_pass_rate"`
ExactFilteredCases int `json:"exact_filtered_cases"`
ExactFilteredPassRate float64 `json:"exact_filtered_pass_rate"`
}
// ToolSearchNegativeTrustMetrics is a same-source diagnostic only. A
// forbidden exposure means the tool that owns an avoid_when sentence appears
// in the ranking; it does not prove that a model would call that tool.
type ToolSearchNegativeTrustMetrics struct {
Cases int `json:"cases"`
ExcludedOverBudget int `json:"excluded_over_budget"`
ForbiddenAt1 float64 `json:"forbidden_at_1"`
ForbiddenAt5 float64 `json:"forbidden_at_5"`
}
// ToolSearchIntegrityMetrics verifies invariants that do not depend on
// relevance labels.
type ToolSearchIntegrityMetrics struct {
ResponsesEvaluated int `json:"responses_evaluated"`
CatalogBindingFailures int `json:"catalog_binding_failures"`
UnknownCandidateCount int `json:"unknown_candidate_count"`
IneligibleCandidateCount int `json:"ineligible_candidate_count"`
ResponseBudgetViolations int `json:"response_budget_violations"`
}
// ToolSearchTrustMetrics keeps contract integrity separate from ranking
// relevance. Search rank is never presented as a probability of safe use.
type ToolSearchTrustMetrics struct {
Identity ToolSearchIdentityTrustMetrics `json:"identity"`
Negative ToolSearchNegativeTrustMetrics `json:"negative"`
Integrity ToolSearchIntegrityMetrics `json:"integrity"`
}
// ToolSearchContextComparison compares byte-level JSON envelopes, not model
// tokens. The ideal navigation path assumes an oracle already knows the right
// product; its selection success is therefore deliberately not estimated.
type ToolSearchContextComparison struct {
ToolCount int `json:"tool_count"`
FullSchemaAllBytes int `json:"full_schema_all_bytes"`
OverviewBytes int `json:"overview_bytes"`
AverageProductBytes float64 `json:"average_product_bytes"`
AverageInspectBytes float64 `json:"average_inspect_bytes"`
AverageIdealSchemaNavigationBytes float64 `json:"average_ideal_schema_navigation_bytes"`
AverageSearchInspectBytes float64 `json:"average_search_inspect_bytes"`
ReductionVsFullSchema float64 `json:"reduction_vs_full_schema"`
ReductionVsIdealNavigation float64 `json:"reduction_vs_ideal_navigation"`
}
// ToolSearchDiagnosticComparison is generated from the exact typed Catalog
// embedded in the binary. It is a diagnostic proxy, not an Agent completion
// benchmark: its intent queries come from reviewed use_when prose and therefore
// are not independently authored qrels.
type ToolSearchDiagnosticComparison struct {
Version string `json:"version"`
Catalog CatalogVersionRef `json:"catalog"`
Algorithm string `json:"algorithm"`
TopK int `json:"top_k"`
IntentProxy ToolSearchRetrievalMetrics `json:"intent_proxy"`
IntentExcludedOverBudget int `json:"intent_excluded_over_budget"`
IntentLanguageSlices map[string]ToolSearchRetrievalMetrics `json:"intent_language_slices"`
WorkflowRaw ToolSearchWorkflowMetrics `json:"workflow_raw"`
WorkflowDecomposed ToolSearchWorkflowMetrics `json:"workflow_decomposed"`
Trust ToolSearchTrustMetrics `json:"trust"`
Context ToolSearchContextComparison `json:"context"`
AgentABStatus string `json:"agent_ab_status"`
Limitations []string `json:"limitations"`
}
// BuildDeliveryToolSearchDiagnosticComparison evaluates the shipped Go
// retriever against the declaration-assembled Catalog without reading a
// generated repository JSON artifact. Results belong under policy-tmp.
func BuildDeliveryToolSearchDiagnosticComparison(ctx context.Context, workflows []ToolSearchWorkflowEvaluationCase) (ToolSearchDiagnosticComparison, error) {
return BuildDeliveryToolSearchDiagnosticComparisonForAlgorithm(ctx, workflows, DefaultToolSearchConfig().LexicalAlgorithm)
}
// BuildDeliveryToolSearchDiagnosticComparisonForAlgorithm runs one shipped Go
// lexical implementation over exactly the same typed Catalog and fixtures.
// This keeps shadow algorithm comparison out of the production default.
func BuildDeliveryToolSearchDiagnosticComparisonForAlgorithm(ctx context.Context, workflows []ToolSearchWorkflowEvaluationCase, algorithm string) (ToolSearchDiagnosticComparison, error) {
if err := deliverySchemaCatalogError(); err != nil {
return ToolSearchDiagnosticComparison{}, fmt.Errorf("assemble typed Schema registry for evaluation: %w", err)
}
loaded := deliverySchemaCatalog()
config := DefaultToolSearchConfig()
config.LexicalAlgorithm = algorithm
config.CatalogSourceHash = loaded.Snapshot.SourceHash
config.CatalogSurfaceHash = loaded.Snapshot.SurfaceHash
engine, err := NewToolSearchEngine(loaded.Registry, config)
if err != nil {
return ToolSearchDiagnosticComparison{}, err
}
allPayload, err := deliverySchemaAllPayload()
if err != nil {
return ToolSearchDiagnosticComparison{}, fmt.Errorf("render full Schema: %w", err)
}
overviewPayload, err := deliverySchemaOverviewPayload()
if err != nil {
return ToolSearchDiagnosticComparison{}, fmt.Errorf("render Schema overview: %w", err)
}
allBytes, err := compactJSONSize(allPayload)
if err != nil {
return ToolSearchDiagnosticComparison{}, err
}
overviewBytes, err := compactJSONSize(stripSchemaPayloadCompact(overviewPayload))
if err != nil {
return ToolSearchDiagnosticComparison{}, err
}
type proxyCase struct {
query string
gold string
product string
}
proxyCases := make([]proxyCase, 0)
intentExcludedOverBudget := 0
for _, product := range loaded.Registry.Products {
for _, tool := range product.Tools {
for _, query := range tool.Selection.UseWhen {
if query != "" && toolSearchQueryWithinBudget(query) {
proxyCases = append(proxyCases, proxyCase{query: query, gold: tool.Identity.CanonicalPath, product: product.ID})
} else if query != "" {
intentExcludedOverBudget++
}
}
}
}
sort.Slice(proxyCases, func(i, j int) bool {
if proxyCases[i].gold == proxyCases[j].gold {
return proxyCases[i].query < proxyCases[j].query
}
return proxyCases[i].gold < proxyCases[j].gold
})
intentAccumulator := &toolSearchRetrievalAccumulator{}
languageAccumulators := map[string]*toolSearchRetrievalAccumulator{}
var totalProductBytes, totalInspectBytes, totalSearchBytes float64
productSizes := make(map[string]int)
inspectSizes := make(map[string]int)
for _, item := range proxyCases {
if err := ctx.Err(); err != nil {
return ToolSearchDiagnosticComparison{}, err
}
response, searchErr := engine.Search(ctx, ToolSearchRequest{Query: item.query, Limit: 5, CandidateLimit: 20})
if searchErr != nil {
return ToolSearchDiagnosticComparison{}, fmt.Errorf("search proxy case %s: %w", item.gold, searchErr)
}
rank := 0
for index, candidate := range response.Candidates {
if candidate.CanonicalPath != item.gold {
continue
}
rank = index + 1
break
}
intentAccumulator.observe(rank, len(response.Candidates) == 0)
slice := toolSearchQueryLanguageSlice(item.query)
if languageAccumulators[slice] == nil {
languageAccumulators[slice] = &toolSearchRetrievalAccumulator{}
}
languageAccumulators[slice].observe(rank, len(response.Candidates) == 0)
encodedResponse, marshalErr := json.Marshal(response)
if marshalErr != nil {
return ToolSearchDiagnosticComparison{}, marshalErr
}
totalSearchBytes += float64(len(encodedResponse))
productBytes, ok := productSizes[item.product]
if !ok {
payload, payloadErr := queryDeliverySchemaPayload([]string{item.product})
if payloadErr != nil {
return ToolSearchDiagnosticComparison{}, payloadErr
}
productBytes, payloadErr = compactJSONSize(stripSchemaPayloadCompact(payload))
if payloadErr != nil {
return ToolSearchDiagnosticComparison{}, payloadErr
}
productSizes[item.product] = productBytes
}
totalProductBytes += float64(productBytes)
inspectBytes, ok := inspectSizes[item.gold]
if !ok {
payload, payloadErr := queryDeliverySchemaPayload([]string{item.gold})
if payloadErr != nil {
return ToolSearchDiagnosticComparison{}, payloadErr
}
payload = stripSchemaPayloadCompact(payload)
inspectBytes, payloadErr = compactJSONSize(SchemaInspectV1Response{
Version: "schema-inspect.v1",
Catalog: CatalogVersionRef{SourceHash: loaded.Snapshot.SourceHash, SurfaceHash: loaded.Snapshot.SurfaceHash},
ToolSpec: payload,
})
if payloadErr != nil {
return ToolSearchDiagnosticComparison{}, payloadErr
}
inspectSizes[item.gold] = inspectBytes
}
totalInspectBytes += float64(inspectBytes)
}
count := float64(len(proxyCases))
intent := intentAccumulator.metrics()
languageSlices := make(map[string]ToolSearchRetrievalMetrics, len(languageAccumulators))
for slice, accumulator := range languageAccumulators {
languageSlices[slice] = accumulator.metrics()
}
contextComparison := ToolSearchContextComparison{
ToolCount: len(loaded.Index.CanonicalPaths()),
FullSchemaAllBytes: allBytes,
OverviewBytes: overviewBytes,
}
if count > 0 {
contextComparison.AverageProductBytes = totalProductBytes / count
contextComparison.AverageInspectBytes = totalInspectBytes / count
contextComparison.AverageIdealSchemaNavigationBytes = float64(overviewBytes) + contextComparison.AverageProductBytes + contextComparison.AverageInspectBytes
contextComparison.AverageSearchInspectBytes = totalSearchBytes/count + contextComparison.AverageInspectBytes
contextComparison.ReductionVsFullSchema = reduction(float64(allBytes), contextComparison.AverageSearchInspectBytes)
contextComparison.ReductionVsIdealNavigation = reduction(contextComparison.AverageIdealSchemaNavigationBytes, contextComparison.AverageSearchInspectBytes)
}
rawWorkflow, decomposedWorkflow, err := evaluateToolSearchWorkflows(ctx, engine, workflows)
if err != nil {
return ToolSearchDiagnosticComparison{}, err
}
trust, err := evaluateToolSearchTrust(ctx, engine, loaded.Registry)
if err != nil {
return ToolSearchDiagnosticComparison{}, err
}
return ToolSearchDiagnosticComparison{
Version: "tool-search-diagnostic-comparison.v1",
Catalog: engine.catalogVersion(),
Algorithm: engine.lexical.Name(),
TopK: 5,
IntentProxy: intent,
IntentExcludedOverBudget: intentExcludedOverBudget,
IntentLanguageSlices: languageSlices,
WorkflowRaw: rawWorkflow,
WorkflowDecomposed: decomposedWorkflow,
Trust: trust,
Context: contextComparison,
AgentABStatus: "not_run_requires_independent_tasks_and_model_runs",
Limitations: []string{
"intent_proxy queries are reviewed use_when prose from the same Catalog and are not independent qrels",
"selection prose that exceeds the production query byte or rune budget is counted and excluded rather than weakening runtime limits",
"direct Schema navigation bytes assume an oracle selects the correct product and do not estimate model selection success",
"search plus inspect bytes also inspect the gold leaf even when retrieval misses and are an oracle-assisted capacity upper bound, not observed Agent cost",
"workflow decomposition uses reviewed fixture subqueries and is an upper-bound diagnostic",
"negative queries are same-source avoid_when prose without alternative gold and measure candidate exposure rather than Agent calls",
"byte counts are compact JSON bytes rather than provider-specific tokenizer counts",
"true Agent task success, unsafe action rate, recovery success, latency, and token cost require paired Agent A/B runs",
},
}, nil
}
type toolSearchRetrievalAccumulator struct {
cases int
hitAt1 int
hitAt5 int
reciprocal float64
zeroResults int
}
func (a *toolSearchRetrievalAccumulator) observe(rank int, zeroResults bool) {
a.cases++
if zeroResults {
a.zeroResults++
}
if rank == 1 {
a.hitAt1++
}
if rank > 0 && rank <= 5 {
a.hitAt5++
a.reciprocal += 1 / float64(rank)
}
}
func (a *toolSearchRetrievalAccumulator) metrics() ToolSearchRetrievalMetrics {
result := ToolSearchRetrievalMetrics{Cases: a.cases}
if a.cases == 0 {
return result
}
denominator := float64(a.cases)
result.RecallAt1 = float64(a.hitAt1) / denominator
result.RecallAt5 = float64(a.hitAt5) / denominator
result.MeanReciprocalRankAt5 = a.reciprocal / denominator
result.ZeroResultRate = float64(a.zeroResults) / denominator
return result
}
func toolSearchQueryLanguageSlice(query string) string {
hasChinese := false
hasASCIIWord := false
for _, character := range query {
if unicode.Is(unicode.Han, character) {
hasChinese = true
}
if character <= unicode.MaxASCII && (unicode.IsLetter(character) || unicode.IsDigit(character)) {
hasASCIIWord = true
}
}
if hasChinese && hasASCIIWord {
return "mixed_chinese_ascii"
}
if hasChinese {
return "chinese_only"
}
return "non_chinese"
}
func evaluateToolSearchTrust(ctx context.Context, engine *ToolSearchEngine, registry SchemaRegistry) (ToolSearchTrustMetrics, error) {
result := ToolSearchTrustMetrics{}
var canonicalPassed, cliPassed, aliasPassed, nfkcPassed, filteredPassed int
var forbiddenAt1, forbiddenAt5 int
for _, product := range registry.Products {
for _, tool := range product.Tools {
if err := ctx.Err(); err != nil {
return result, err
}
result.Identity.CanonicalCases++
response, err := engine.Search(ctx, ToolSearchRequest{Query: tool.Identity.CanonicalPath})
if err != nil {
return result, fmt.Errorf("audit exact canonical %s: %w", tool.Identity.CanonicalPath, err)
}
observeToolSearchIntegrity(&result.Integrity, engine, response)
if exactSearchMatches(response, tool.Identity.CanonicalPath) {
canonicalPassed++
}
if cliPath := tool.Identity.PrimaryCLIPath; cliPath != "" {
result.Identity.PrimaryCLICases++
response, err = engine.Search(ctx, ToolSearchRequest{Query: cliPath})
if err != nil {
return result, fmt.Errorf("audit exact CLI %s: %w", cliPath, err)
}
observeToolSearchIntegrity(&result.Integrity, engine, response)
if exactSearchMatches(response, tool.Identity.CanonicalPath) {
cliPassed++
}
}
for _, alias := range tool.Identity.Aliases {
result.Identity.AliasCases++
response, err = engine.Search(ctx, ToolSearchRequest{Query: alias})
if err != nil {
return result, fmt.Errorf("audit exact alias %s: %w", alias, err)
}
observeToolSearchIntegrity(&result.Integrity, engine, response)
if exactSearchMatches(response, tool.Identity.CanonicalPath) {
aliasPassed++
}
}
nfkcQuery := toolSearchFullwidthASCII(tool.Identity.CanonicalPath)
if nfkcQuery != tool.Identity.CanonicalPath {
result.Identity.NFKCCases++
response, err = engine.Search(ctx, ToolSearchRequest{Query: nfkcQuery})
if err != nil {
return result, fmt.Errorf("audit NFKC identity %s: %w", tool.Identity.CanonicalPath, err)
}
observeToolSearchIntegrity(&result.Integrity, engine, response)
if exactSearchMatches(response, tool.Identity.CanonicalPath) {
nfkcPassed++
}
}
result.Identity.ExactFilteredCases++
response, err = engine.Search(ctx, ToolSearchRequest{
Query: tool.Identity.CanonicalPath,
ExcludeCanonicalPaths: []string{tool.Identity.CanonicalPath},
})
if err != nil {
return result, fmt.Errorf("audit exact filtered %s: %w", tool.Identity.CanonicalPath, err)
}
observeToolSearchIntegrity(&result.Integrity, engine, response)
if response.Strategy == "exact_filtered" && response.Abstained && len(response.Candidates) == 0 && response.ExactFiltered != nil && response.ExactFiltered.CanonicalPath == tool.Identity.CanonicalPath && response.ExactFiltered.Reason == "excluded" {
filteredPassed++
}
for _, query := range tool.Selection.AvoidWhen {
if query == "" {
continue
}
if !toolSearchQueryWithinBudget(query) {
result.Negative.ExcludedOverBudget++
continue
}
result.Negative.Cases++
response, err = engine.Search(ctx, ToolSearchRequest{Query: query, Limit: 5, CandidateLimit: 20})
if err != nil {
return result, fmt.Errorf("audit negative %s: %w", tool.Identity.CanonicalPath, err)
}
observeToolSearchIntegrity(&result.Integrity, engine, response)
for rank, candidate := range response.Candidates {
if candidate.CanonicalPath != tool.Identity.CanonicalPath {
continue
}
if rank == 0 {
forbiddenAt1++
}
forbiddenAt5++
break
}
}
}
}
if result.Identity.CanonicalCases > 0 {
result.Identity.CanonicalPassRate = float64(canonicalPassed) / float64(result.Identity.CanonicalCases)
}
if result.Identity.PrimaryCLICases > 0 {
result.Identity.PrimaryCLIPassRate = float64(cliPassed) / float64(result.Identity.PrimaryCLICases)
}
if result.Identity.AliasCases > 0 {
result.Identity.AliasPassRate = float64(aliasPassed) / float64(result.Identity.AliasCases)
}
if result.Identity.NFKCCases > 0 {
result.Identity.NFKCPassRate = float64(nfkcPassed) / float64(result.Identity.NFKCCases)
}
if result.Identity.ExactFilteredCases > 0 {
result.Identity.ExactFilteredPassRate = float64(filteredPassed) / float64(result.Identity.ExactFilteredCases)
}
if result.Negative.Cases > 0 {
result.Negative.ForbiddenAt1 = float64(forbiddenAt1) / float64(result.Negative.Cases)
result.Negative.ForbiddenAt5 = float64(forbiddenAt5) / float64(result.Negative.Cases)
}
return result, nil
}
func toolSearchFullwidthASCII(value string) string {
runes := []rune(value)
for index, character := range runes {
if character >= '!' && character <= '~' {
runes[index] = character + 0xFEE0
}
}
return string(runes)
}
func toolSearchQueryWithinBudget(query string) bool {
return len(query) <= maxToolSearchQueryBytes && utf8.RuneCountInString(query) <= maxToolSearchQueryRunes
}
func exactSearchMatches(response ToolSearchResponse, canonical string) bool {
return response.Strategy == "exact_guard" && len(response.Candidates) == 1 && response.Candidates[0].CanonicalPath == canonical && response.Candidates[0].RequiresInspect
}
func observeToolSearchIntegrity(metrics *ToolSearchIntegrityMetrics, engine *ToolSearchEngine, response ToolSearchResponse) {
metrics.ResponsesEvaluated++
if response.Catalog != engine.catalogVersion() {
metrics.CatalogBindingFailures++
}
if encoded, err := json.Marshal(response); err != nil || len(encoded) > maxToolSearchResponseBytes {
metrics.ResponseBudgetViolations++
}
for _, candidate := range response.Candidates {
document, ok := engine.documents[candidate.CanonicalPath]
if !ok {
metrics.UnknownCandidateCount++
continue
}
if !toolSearchEligible(document.tool, nil, nil, nil) {
metrics.IneligibleCandidateCount++
}
}
}
func evaluateToolSearchWorkflows(ctx context.Context, engine *ToolSearchEngine, workflows []ToolSearchWorkflowEvaluationCase) (ToolSearchWorkflowMetrics, ToolSearchWorkflowMetrics, error) {
raw := ToolSearchWorkflowMetrics{Cases: len(workflows)}
decomposed := ToolSearchWorkflowMetrics{Cases: len(workflows)}
if len(workflows) == 0 {
return raw, decomposed, nil
}
for _, workflow := range workflows {
if workflow.ID == "" || workflow.Query == "" || len(workflow.Required) == 0 {
return raw, decomposed, fmt.Errorf("workflow evaluation case must have id, query and required tools")
}
one, err := engine.Search(ctx, ToolSearchRequest{Query: workflow.Query, Limit: 5, CandidateLimit: 20})
if err != nil {
return raw, decomposed, fmt.Errorf("search workflow %s: %w", workflow.ID, err)
}
many, err := engine.SearchSubqueries(ctx, workflow.Subqueries, ToolSearchRequest{Query: workflow.Query, Limit: 5, CandidateLimit: 20})
if err != nil {
return raw, decomposed, fmt.Errorf("search decomposed workflow %s: %w", workflow.ID, err)
}
accumulateWorkflowMetrics(&raw, one.Candidates, workflow.Required)
accumulateWorkflowMetrics(&decomposed, many.Candidates, workflow.Required)
}
denominator := float64(len(workflows))
raw.CompleteAt5 /= denominator
raw.RequiredRecallAt5 /= denominator
decomposed.CompleteAt5 /= denominator
decomposed.RequiredRecallAt5 /= denominator
return raw, decomposed, nil
}
func accumulateWorkflowMetrics(metrics *ToolSearchWorkflowMetrics, candidates []ToolReference, required []string) {
found := make(map[string]bool, len(candidates))
for _, candidate := range candidates {
found[candidate.CanonicalPath] = true
}
hits := 0
for _, canonical := range required {
if found[canonical] {
hits++
}
}
metrics.RequiredRecallAt5 += float64(hits) / float64(len(required))
if hits == len(required) {
metrics.CompleteAt5++
}
}
func compactJSONSize(value any) (int, error) {
encoded, err := json.Marshal(value)
if err != nil {
return 0, fmt.Errorf("marshal compact JSON: %w", err)
}
return len(encoded), nil
}
func reduction(baseline, candidate float64) float64 {
if baseline <= 0 {
return 0
}
return math.Max(-1, math.Min(1, 1-candidate/baseline))
}
@@ -0,0 +1,98 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package cli
import "testing"
func TestAggregateToolSearchAgentAB(t *testing.T) {
catalog := CatalogVersionRef{SourceHash: "sha256:source", SurfaceHash: "sha256:surface"}
input := ToolSearchAgentABInput{
Version: "tool-search-agent-ab.v1",
Catalog: catalog,
Runs: []ToolSearchAgentABRun{
{CaseID: "a", Trial: 0, Arm: ToolSearchAgentArmDirectSchema, CorrectToolPlan: true, ContextTokens: 100, ToolCalls: 3, LatencyMS: 1000},
{CaseID: "a", Trial: 0, Arm: ToolSearchAgentArmSearchInspect, TaskCompleted: true, CorrectToolPlan: true, ContextTokens: 40, ToolCalls: 2, LatencyMS: 800},
{CaseID: "b", Trial: 0, Arm: ToolSearchAgentArmDirectSchema, TaskCompleted: true, CorrectToolPlan: true, UnsafeAction: true, RecoveryAttempted: true, ContextTokens: 120, ToolCalls: 4, LatencyMS: 1200},
{CaseID: "b", Trial: 0, Arm: ToolSearchAgentArmSearchInspect, TaskCompleted: true, CorrectToolPlan: true, RecoveryAttempted: true, RecoverySucceeded: true, ContextTokens: 50, ToolCalls: 3, LatencyMS: 900},
},
}
report, err := AggregateToolSearchAgentAB(input)
if err != nil {
t.Fatalf("AggregateToolSearchAgentAB() error = %v", err)
}
if report.Cases != 2 || report.PairedTrials != 2 {
t.Fatalf("pair counts = cases %d trials %d", report.Cases, report.PairedTrials)
}
if report.DirectSchema.TaskSuccessRate != 0.5 || report.SearchInspect.TaskSuccessRate != 1 {
t.Fatalf("unexpected success rates: direct=%v search=%v", report.DirectSchema.TaskSuccessRate, report.SearchInspect.TaskSuccessRate)
}
if delta := report.Deltas["context_tokens"].Delta; delta != -65 {
t.Fatalf("context delta = %v, want -65", delta)
}
if _, ok := report.Deltas["recovery_success_rate_paired_attempts"]; !ok {
t.Fatal("paired recovery delta missing")
}
}
func TestAggregateToolSearchAgentABRejectsUnpairedRuns(t *testing.T) {
_, err := AggregateToolSearchAgentAB(ToolSearchAgentABInput{
Version: "tool-search-agent-ab.v1",
Catalog: CatalogVersionRef{SourceHash: "source", SurfaceHash: "surface"},
Runs: []ToolSearchAgentABRun{
{CaseID: "a", Arm: ToolSearchAgentArmDirectSchema},
},
})
if err == nil {
t.Fatal("AggregateToolSearchAgentAB() accepted an unpaired run")
}
}
func TestScoreToolSearchAgentPlanningAB(t *testing.T) {
workflows := []ToolSearchWorkflowEvaluationCase{
{ID: "one", Required: []string{"chat.search_groups", "chat.send_personal_message"}},
{ID: "two", Required: []string{"drive.upload", "drive.add_permission"}},
}
direct := ToolSearchAgentPlanOutput{Results: []ToolSearchAgentPlanResult{
{ID: "one", CanonicalPaths: []string{"chat.search_groups", "chat.send_personal_message"}},
{ID: "two", CanonicalPaths: []string{"drive.upload", "drive.add_permission"}},
}}
search := ToolSearchAgentPlanOutput{Results: []ToolSearchAgentPlanResult{
{ID: "one", CanonicalPaths: []string{"chat.list_owned_or_admin_groups", "chat.search_groups", "chat.send_personal_message"}},
{ID: "two", CanonicalPaths: []string{"drive.upload", "drive.add_permission"}},
}}
tools := []ToolSpec{
toolSearchTestTool("chat", "search_groups", "chat search-groups", "搜索群聊", "read"),
toolSearchTestTool("chat", "send_personal_message", "chat message send", "发送消息", "write"),
toolSearchTestTool("drive", "upload", "drive upload", "上传文件", "write"),
toolSearchTestTool("drive", "add_permission", "drive permission add", "添加权限", "write"),
toolSearchTestTool("chat", "list_owned_or_admin_groups", "chat list-groups", "列出群聊", "read"),
}
index, indexErr := (SchemaRegistry{Kind: "schema", Level: "catalog", Products: []ProductSpec{
{ID: "chat", Tools: []ToolSpec{tools[0], tools[1], tools[4]}},
{ID: "drive", Tools: tools[2:4]},
}}).Index()
if indexErr != nil {
t.Fatal(indexErr)
}
report, err := scoreToolSearchAgentPlanningAB(index, "test-model", workflows, direct, search)
if err != nil {
t.Fatalf("ScoreToolSearchAgentPlanningAB() error = %v", err)
}
if report.DirectSchema.ExactMinimalRate != 1 || report.SearchInspect.ExactMinimalRate != 0.5 {
t.Fatalf("minimal rates = direct %v search %v", report.DirectSchema.ExactMinimalRate, report.SearchInspect.ExactMinimalRate)
}
if report.SearchInspect.CompleteRate != 1 || report.SearchInspect.UnnecessaryTools != 1 {
t.Fatalf("search metrics = %+v", report.SearchInspect)
}
}
@@ -0,0 +1,357 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
package cli
import (
"context"
"fmt"
"math"
"math/rand"
"sort"
"strings"
)
// The product-cluster paired bootstrap uses a fixed sample count and seed so a
// sealed qrels run stays reproducible; both are published in the report so an
// auditor does not have to read them out of the source.
const (
toolSearchIndependentBootstrapSamples = 10_000
toolSearchIndependentBootstrapSeed = 20260813
)
// ToolSearchIndependentQrel is an independently authored graded relevance
// judgment. Relevance uses the frozen 1..3 qrels scale.
type ToolSearchIndependentQrel struct {
Canonical string `json:"canonical"`
Relevance int `json:"relevance"`
}
type ToolSearchIndependentWorkflow struct {
Subqueries []string `json:"subqueries,omitempty"`
Required []string `json:"required"`
}
type ToolSearchIndependentCase struct {
ID string `json:"id"`
Query string `json:"query"`
Language string `json:"language"`
Qrels []ToolSearchIndependentQrel `json:"qrels"`
Forbidden []string `json:"forbidden,omitempty"`
AlternativeGold []string `json:"alternative_gold,omitempty"`
Workflow *ToolSearchIndependentWorkflow `json:"workflow,omitempty"`
ConfusionFamily []string `json:"confusion_family,omitempty"`
}
type ToolSearchIndependentRankingMetrics struct {
Cases int `json:"cases"`
RecallAt1 float64 `json:"recall_at_1"`
RecallAt5 float64 `json:"recall_at_5"`
MeanReciprocalRankAt5 float64 `json:"mean_reciprocal_rank_at_5"`
NDCGAt5 float64 `json:"ndcg_at_5"`
ZeroResultRate float64 `json:"zero_result_rate"`
}
type ToolSearchIndependentSafetyMetrics struct {
ForbiddenCases int `json:"forbidden_cases"`
ForbiddenExposureAt5 float64 `json:"forbidden_exposure_at_5"`
AlternativeRecallAt5 float64 `json:"alternative_recall_at_5"`
SiblingConfusionCases int `json:"sibling_confusion_cases"`
SiblingExposureAt5 float64 `json:"sibling_exposure_at_5"`
}
type ToolSearchIndependentReport struct {
Version string `json:"version"`
Catalog CatalogVersionRef `json:"catalog"`
Algorithm string `json:"algorithm"`
ControlAlgorithm string `json:"control_algorithm"`
Overall ToolSearchIndependentRankingMetrics `json:"overall"`
ControlOverall ToolSearchIndependentRankingMetrics `json:"control_overall"`
RecallAt5Delta float64 `json:"recall_at_5_delta"`
ProductClusterCI95 ToolSearchConfidenceInterval `json:"product_cluster_recall_at_5_delta_ci_95"`
LanguageSlices map[string]ToolSearchIndependentRankingMetrics `json:"language_slices"`
Safety ToolSearchIndependentSafetyMetrics `json:"safety"`
Workflow ToolSearchWorkflowMetrics `json:"workflow"`
// BootstrapSamples/BootstrapSeed are set only by the paired delivery
// evaluation that computes ProductClusterCI95; they make that interval
// reproducible without reading the source.
BootstrapSamples int `json:"bootstrap_samples,omitempty"`
BootstrapSeed int64 `json:"bootstrap_seed,omitempty"`
}
type ToolSearchConfidenceInterval struct {
Lower float64 `json:"lower"`
Upper float64 `json:"upper"`
}
type independentRankingAccumulator struct {
metrics ToolSearchIndependentRankingMetrics
recall1 float64
recall5 float64
mrr5 float64
ndcg5 float64
zero int
}
// BuildDeliveryToolSearchIndependentEvaluation evaluates the action-ranking
// candidate against the shipped fielded BM25 control on sealed external qrels.
// It never tunes parameters or reads Catalog JSON; the caller owns qrels
// sealing and threshold enforcement. The candidate may become the runtime
// default only after the independently frozen default-switch gate passes.
func BuildDeliveryToolSearchIndependentEvaluation(ctx context.Context, cases []ToolSearchIndependentCase) (ToolSearchIndependentReport, error) {
if err := deliverySchemaCatalogError(); err != nil {
return ToolSearchIndependentReport{}, err
}
loaded := deliverySchemaCatalog()
candidateConfig := DefaultToolSearchConfig()
candidateConfig.LexicalAlgorithm = ToolSearchLexicalBM25Action
candidateConfig.CatalogSourceHash = loaded.Snapshot.SourceHash
candidateConfig.CatalogSurfaceHash = loaded.Snapshot.SurfaceHash
engine, err := NewToolSearchEngine(loaded.Registry, candidateConfig)
if err != nil {
return ToolSearchIndependentReport{}, err
}
report, err := EvaluateToolSearchIndependent(ctx, engine, cases)
if err != nil {
return ToolSearchIndependentReport{}, err
}
controlConfig := DefaultToolSearchConfig()
controlConfig.CatalogSourceHash = loaded.Snapshot.SourceHash
controlConfig.CatalogSurfaceHash = loaded.Snapshot.SurfaceHash
control, err := NewToolSearchEngine(loaded.Registry, controlConfig)
if err != nil {
return ToolSearchIndependentReport{}, err
}
controlReport, err := EvaluateToolSearchIndependent(ctx, control, cases)
if err != nil {
return ToolSearchIndependentReport{}, err
}
report.ControlAlgorithm = control.lexical.Name()
report.ControlOverall = controlReport.Overall
report.RecallAt5Delta = report.Overall.RecallAt5 - report.ControlOverall.RecallAt5
report.ProductClusterCI95, err = pairedProductClusterRecallCI(ctx, engine, control, cases, toolSearchIndependentBootstrapSamples, toolSearchIndependentBootstrapSeed)
if err != nil {
return ToolSearchIndependentReport{}, err
}
report.BootstrapSamples = toolSearchIndependentBootstrapSamples
report.BootstrapSeed = toolSearchIndependentBootstrapSeed
return report, nil
}
func EvaluateToolSearchIndependent(ctx context.Context, engine *ToolSearchEngine, cases []ToolSearchIndependentCase) (ToolSearchIndependentReport, error) {
if engine == nil || len(cases) == 0 {
return ToolSearchIndependentReport{}, fmt.Errorf("independent evaluation requires an engine and non-empty cases")
}
overall := &independentRankingAccumulator{}
slices := map[string]*independentRankingAccumulator{}
var forbiddenCases, forbiddenExposed int
var siblingCases, siblingExposed int
var alternativeRecall float64
workflowCases := 0
workflowComplete := 0
workflowRequiredRecall := 0.0
for _, item := range cases {
if err := ctx.Err(); err != nil {
return ToolSearchIndependentReport{}, err
}
response, searchErr := engine.Search(ctx, ToolSearchRequest{Query: item.Query, Limit: 5, CandidateLimit: 20})
if searchErr != nil {
return ToolSearchIndependentReport{}, fmt.Errorf("evaluate %s: %w", item.ID, searchErr)
}
overall.observe(response.Candidates, item.Qrels)
if slices[item.Language] == nil {
slices[item.Language] = &independentRankingAccumulator{}
}
slices[item.Language].observe(response.Candidates, item.Qrels)
if len(item.Forbidden) > 0 {
forbiddenCases++
if candidatesContainAny(response.Candidates, item.Forbidden) {
forbiddenExposed++
}
alternativeRecall += candidateSetRecall(response.Candidates, item.AlternativeGold)
}
if len(item.ConfusionFamily) > 0 {
siblingCases++
if candidatesContainAny(response.Candidates, item.ConfusionFamily) {
siblingExposed++
}
}
if item.Workflow != nil {
workflowCases++
workflowResponse := response
if len(item.Workflow.Subqueries) > 0 {
workflowResponse, searchErr = engine.SearchSubqueries(ctx, item.Workflow.Subqueries, ToolSearchRequest{Query: item.Query, Limit: 5, CandidateLimit: 20})
if searchErr != nil {
return ToolSearchIndependentReport{}, fmt.Errorf("evaluate workflow %s: %w", item.ID, searchErr)
}
}
recall := candidateSetRecall(workflowResponse.Candidates, item.Workflow.Required)
workflowRequiredRecall += recall
if recall == 1 {
workflowComplete++
}
}
}
report := ToolSearchIndependentReport{
Version: "tool-search-independent-evaluation.v1", Catalog: engine.catalogVersion(), Algorithm: engine.lexical.Name(),
Overall: overall.result(), LanguageSlices: map[string]ToolSearchIndependentRankingMetrics{},
Safety: ToolSearchIndependentSafetyMetrics{ForbiddenCases: forbiddenCases, SiblingConfusionCases: siblingCases},
Workflow: ToolSearchWorkflowMetrics{Cases: workflowCases},
}
for name, accumulator := range slices {
report.LanguageSlices[name] = accumulator.result()
}
if forbiddenCases > 0 {
report.Safety.ForbiddenExposureAt5 = float64(forbiddenExposed) / float64(forbiddenCases)
report.Safety.AlternativeRecallAt5 = alternativeRecall / float64(forbiddenCases)
}
if siblingCases > 0 {
report.Safety.SiblingExposureAt5 = float64(siblingExposed) / float64(siblingCases)
}
if workflowCases > 0 {
report.Workflow.CompleteAt5 = float64(workflowComplete) / float64(workflowCases)
report.Workflow.RequiredRecallAt5 = workflowRequiredRecall / float64(workflowCases)
}
return report, nil
}
func pairedProductClusterRecallCI(ctx context.Context, candidate, control *ToolSearchEngine, cases []ToolSearchIndependentCase, iterations int, seed int64) (ToolSearchConfidenceInterval, error) {
clusters := map[string][]float64{}
for _, item := range cases {
candidateResponse, err := candidate.Search(ctx, ToolSearchRequest{Query: item.Query, Limit: 5, CandidateLimit: 20})
if err != nil {
return ToolSearchConfidenceInterval{}, err
}
controlResponse, err := control.Search(ctx, ToolSearchRequest{Query: item.Query, Limit: 5, CandidateLimit: 20})
if err != nil {
return ToolSearchConfidenceInterval{}, err
}
product := "unknown"
if len(item.Qrels) > 0 {
if split := strings.SplitN(item.Qrels[0].Canonical, ".", 2); len(split) > 0 {
product = split[0]
}
}
clusters[product] = append(clusters[product], candidateSetRecall(candidateResponse.Candidates, qrelCanonicals(item.Qrels))-candidateSetRecall(controlResponse.Candidates, qrelCanonicals(item.Qrels)))
}
clusterNames := make([]string, 0, len(clusters))
clusterMeans := map[string]float64{}
for name, values := range clusters {
clusterNames = append(clusterNames, name)
for _, value := range values {
clusterMeans[name] += value
}
clusterMeans[name] /= float64(len(values))
}
sort.Strings(clusterNames)
if len(clusterNames) == 0 || iterations <= 0 {
return ToolSearchConfidenceInterval{}, fmt.Errorf("paired product bootstrap requires cases and iterations")
}
random := rand.New(rand.NewSource(seed)) // #nosec G404 -- deterministic evaluation bootstrap
values := make([]float64, iterations)
for iteration := range values {
for sample := 0; sample < len(clusterNames); sample++ {
values[iteration] += clusterMeans[clusterNames[random.Intn(len(clusterNames))]]
}
values[iteration] /= float64(len(clusterNames))
}
sort.Float64s(values)
return ToolSearchConfidenceInterval{Lower: percentile(values, 0.025), Upper: percentile(values, 0.975)}, nil
}
func qrelCanonicals(qrels []ToolSearchIndependentQrel) []string {
values := make([]string, 0, len(qrels))
for _, qrel := range qrels {
values = append(values, qrel.Canonical)
}
return values
}
func (a *independentRankingAccumulator) observe(candidates []ToolReference, qrels []ToolSearchIndependentQrel) {
a.metrics.Cases++
if len(candidates) == 0 {
a.zero++
}
relevance := make(map[string]int, len(qrels))
ideal := make([]int, 0, len(qrels))
for _, qrel := range qrels {
relevance[qrel.Canonical] = qrel.Relevance
ideal = append(ideal, qrel.Relevance)
}
found := 0
first := 0
dcg := 0.0
for rank, candidate := range candidates {
if rank >= 5 {
break
}
rel := relevance[candidate.CanonicalPath]
if rel == 0 {
continue
}
found++
if first == 0 {
first = rank + 1
}
dcg += (math.Pow(2, float64(rel)) - 1) / math.Log2(float64(rank+2))
}
if len(candidates) > 0 && relevance[candidates[0].CanonicalPath] > 0 {
a.recall1 += 1 / float64(len(qrels))
}
if len(qrels) > 0 {
a.recall5 += float64(found) / float64(len(qrels))
}
if first > 0 {
a.mrr5 += 1 / float64(first)
}
sort.Sort(sort.Reverse(sort.IntSlice(ideal)))
idcg := 0.0
for rank, rel := range ideal {
if rank >= 5 {
break
}
idcg += (math.Pow(2, float64(rel)) - 1) / math.Log2(float64(rank+2))
}
if idcg > 0 {
a.ndcg5 += dcg / idcg
}
}
func (a *independentRankingAccumulator) result() ToolSearchIndependentRankingMetrics {
result := a.metrics
if result.Cases == 0 {
return result
}
denominator := float64(result.Cases)
result.RecallAt1 = a.recall1 / denominator
result.RecallAt5 = a.recall5 / denominator
result.MeanReciprocalRankAt5 = a.mrr5 / denominator
result.NDCGAt5 = a.ndcg5 / denominator
result.ZeroResultRate = float64(a.zero) / denominator
return result
}
func candidatesContainAny(candidates []ToolReference, paths []string) bool {
for _, candidate := range candidates {
if stringSliceContains(paths, candidate.CanonicalPath) {
return true
}
}
return false
}
func candidateSetRecall(candidates []ToolReference, required []string) float64 {
if len(required) == 0 {
return 0
}
found := 0
for _, canonical := range required {
for _, candidate := range candidates {
if candidate.CanonicalPath == canonical {
found++
break
}
}
}
return float64(found) / float64(len(required))
}
@@ -0,0 +1,70 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
package cli
import (
"context"
"math"
"testing"
)
func TestEvaluateToolSearchIndependentScoresGradedSafetyAndWorkflow(t *testing.T) {
engine := newToolSearchTestEngine(t)
cases := []ToolSearchIndependentCase{
{
ID: "chat", Query: "给群里发送消息", Language: "chinese_only",
Qrels: []ToolSearchIndependentQrel{{Canonical: "chat.send", Relevance: 3}},
Forbidden: []string{"doc.read"}, AlternativeGold: []string{"chat.send"},
ConfusionFamily: []string{"chat.read_status"},
},
{
ID: "workflow", Query: "发送消息并读取文档", Language: "chinese_only",
Qrels: []ToolSearchIndependentQrel{{Canonical: "chat.send", Relevance: 3}, {Canonical: "doc.read", Relevance: 2}},
Workflow: &ToolSearchIndependentWorkflow{
Subqueries: []string{"群聊发送消息", "读取在线文档"},
Required: []string{"chat.send", "doc.read"},
},
},
}
report, err := EvaluateToolSearchIndependent(context.Background(), engine, cases)
if err != nil {
t.Fatal(err)
}
if report.Version != "tool-search-independent-evaluation.v1" || report.Overall.Cases != 2 {
t.Fatalf("report = %#v", report)
}
if report.Overall.RecallAt5 != 1 || report.Overall.NDCGAt5 <= 0 || report.Overall.MeanReciprocalRankAt5 <= 0 {
t.Fatalf("ranking = %#v", report.Overall)
}
if report.Safety.ForbiddenCases != 1 || report.Safety.ForbiddenExposureAt5 != 0 || report.Safety.AlternativeRecallAt5 != 1 ||
report.Safety.SiblingConfusionCases != 1 || report.Safety.SiblingExposureAt5 != 1 {
t.Fatalf("safety = %#v", report.Safety)
}
if report.Workflow.Cases != 1 || report.Workflow.CompleteAt5 != 1 || report.Workflow.RequiredRecallAt5 != 1 {
t.Fatalf("workflow = %#v", report.Workflow)
}
}
func TestIndependentRankingMetricsUseGradedRankDiscounts(t *testing.T) {
accumulator := &independentRankingAccumulator{}
accumulator.observe([]ToolReference{
{CanonicalPath: "irrelevant"},
{CanonicalPath: "relevant.low"},
}, []ToolSearchIndependentQrel{
{Canonical: "relevant.high", Relevance: 3},
{Canonical: "relevant.low", Relevance: 2},
})
metrics := accumulator.result()
wantNDCG := (3 / math.Log2(3)) / (7 + 3/math.Log2(3))
if metrics.RecallAt1 != 0 || metrics.RecallAt5 != 0.5 || metrics.MeanReciprocalRankAt5 != 0.5 || math.Abs(metrics.NDCGAt5-wantNDCG) > 1e-12 || metrics.ZeroResultRate != 0 {
t.Fatalf("metrics = %#v, want R1=0 R5=.5 MRR5=.5 NDCG5=%.12f", metrics, wantNDCG)
}
zero := &independentRankingAccumulator{}
zero.observe(nil, []ToolSearchIndependentQrel{{Canonical: "relevant", Relevance: 3}})
zeroMetrics := zero.result()
if zeroMetrics.ZeroResultRate != 1 || zeroMetrics.RecallAt5 != 0 || zeroMetrics.NDCGAt5 != 0 {
t.Fatalf("zero metrics = %#v", zeroMetrics)
}
}
+662
View File
@@ -0,0 +1,662 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
package cli
import (
"context"
"fmt"
"math"
"regexp"
"sort"
"strings"
"unicode/utf8"
)
const (
ToolSearchLexicalBM25 = "fielded_bm25_ensemble"
ToolSearchLexicalBM25Action = "fielded_bm25_action_v1"
ToolSearchLexicalTFIDF = "fielded_tfidf_cosine"
)
// ToolSearchLexicalRequest contains a prefiltered Catalog domain. Hard
// filtering occurs before scoring, so a retriever cannot spend its Top-K on
// candidates that DWS is required to reject.
type ToolSearchLexicalRequest struct {
Query string
CandidateLimit int
EligibleCanonicalPaths []string
}
// LexicalHit is an internal ordering result, not a confidence probability.
// Explain is populated only when the engine was built with Explain=true.
type LexicalHit struct {
CanonicalPath string
Score float64
MatchedFields []string
Explain *ToolSearchScoreBreakdown
}
// LexicalRetriever is the zero-model local recall boundary. Implementations
// must return a deterministic score-desc/canonical-asc order and omit zero
// score documents.
type LexicalRetriever interface {
Name() string
Retrieve(context.Context, ToolSearchLexicalRequest) ([]LexicalHit, error)
}
func newToolSearchLexicalRetriever(documents map[string]toolSearchDocument, config ToolSearchConfig) (LexicalRetriever, error) {
var retriever LexicalRetriever
switch config.LexicalAlgorithm {
case ToolSearchLexicalBM25Action:
retriever = newToolSearchActionRetriever(
newToolSearchBM25Retriever(documents, config.FieldWeights, config.BM25K1, config.BM25B, config.Explain),
documents,
config.Explain,
)
case ToolSearchLexicalBM25:
retriever = newToolSearchBM25Retriever(documents, config.FieldWeights, config.BM25K1, config.BM25B, config.Explain)
case ToolSearchLexicalTFIDF:
retriever = newToolSearchTFIDFRetriever(documents, config.FieldWeights, config.Explain)
default:
return nil, fmt.Errorf("unknown tool search lexical algorithm %q", config.LexicalAlgorithm)
}
return newToolSearchAvoidWhenRetriever(retriever, documents, config.Explain), nil
}
type toolSearchActionRetriever struct {
base LexicalRetriever
documents map[string]toolSearchDocument
explain bool
}
var toolSearchASCIIWordPattern = regexp.MustCompile(`[A-Za-z][A-Za-z0-9._-]*`)
func newToolSearchActionRetriever(base LexicalRetriever, documents map[string]toolSearchDocument, explain bool) *toolSearchActionRetriever {
return &toolSearchActionRetriever{base: base, documents: documents, explain: explain}
}
func (*toolSearchActionRetriever) Name() string { return ToolSearchLexicalBM25Action }
func (r *toolSearchActionRetriever) Retrieve(ctx context.Context, request ToolSearchLexicalRequest) ([]LexicalHit, error) {
deep := request
deep.CandidateLimit = len(request.EligibleCanonicalPaths)
hits, err := r.base.Retrieve(ctx, deep)
if err != nil {
return nil, err
}
query := classifyToolSearchQuery(request.Query)
if !query.hasSignal() || toolSearchHasUnclassifiedASCII(request.Query) {
return truncateAndSortLexicalHits(hits, request.CandidateLimit), nil
}
for index := range hits {
multiplier := toolSearchStructuredMultiplier(query, r.documents[hits[index].CanonicalPath].tool)
hits[index].Score *= multiplier
if r.explain && hits[index].Explain != nil {
hits[index].Explain.Multiplier = &multiplier
hits[index].Explain.Score = hits[index].Score
hits[index].Explain.QueryClass = newToolSearchExplainQueryClass(query)
}
}
return truncateAndSortLexicalHits(hits, request.CandidateLimit), nil
}
// toolSearchHasUnclassifiedASCII reports whether the query carries a technical
// identifier (camelCase or separator-joined tokens such as openConversationId,
// task_id, file-path) that the compact vocabularies cannot classify. Plain
// English words no longer disable structured reranking: they match (or fail
// to match) the vocabularies exactly like Chinese phrases do, so mixed and
// English queries keep the effect/product/action/entity routing instead of
// silently degrading to raw BM25.
func toolSearchHasUnclassifiedASCII(query string) bool {
for _, token := range toolSearchASCIIWordPattern.FindAllString(query, -1) {
if len(token) < 4 {
continue
}
switch strings.ToLower(token) {
case "id", "ids", "url", "uri":
continue
}
if strings.ContainsAny(token, "._-") {
return true
}
// Only an internal uppercase transition (openConversationId) marks a
// technical identifier; a capitalized plain word ("Send") does not.
if internal := token[1:]; strings.ToLower(internal) != internal {
return true
}
}
return false
}
type toolSearchQueryClass struct {
actions map[string]bool
products map[string]bool
entities map[string]bool
effect string
}
func (q toolSearchQueryClass) hasSignal() bool {
return len(q.actions) > 0 || len(q.products) > 0 || len(q.entities) > 0 || q.effect != ""
}
type toolSearchTermClass struct {
name string
query []string
tool []string
effect string
}
var toolSearchActionClasses = []toolSearchTermClass{
{name: "approve", query: []string{"同意", "批准", "审批通过", "approve"}, tool: []string{"approve"}},
{name: "reject", query: []string{"拒绝", "驳回", "reject"}, tool: []string{"reject"}},
{name: "upload", query: []string{"上传", "upload"}, tool: []string{"upload"}},
{name: "download", query: []string{"下载", "download"}, tool: []string{"download"}},
{name: "send", query: []string{"发送", "发给", "发消息", "通知", "投递", "send", "notify"}, tool: []string{"send", "notify"}},
{name: "search", query: []string{"搜索", "查找", "检索", "定位", "search", "find", "locate"}, tool: []string{"search", "find", "resolve"}},
{name: "list", query: []string{"列出", "列表", "浏览", "遍历", "list", "browse"}, tool: []string{"list", "all"}},
{name: "read", query: []string{"读取", "查看", "获取", "详情", "正文", "read", "fetch"}, tool: []string{"read", "get", "detail", "info", "query"}},
{name: "create", query: []string{"创建", "新建", "create"}, tool: []string{"create", "new"}},
{name: "add", query: []string{"添加", "追加", "增加", "授权", "授予", "add", "invite"}, tool: []string{"add", "grant", "invite"}},
{name: "update", query: []string{"更新", "修改", "编辑", "重命名", "调整", "update", "edit", "rename"}, tool: []string{"update", "edit", "rename", "set"}},
{name: "delete", query: []string{"删除", "移除", "清除", "delete", "remove"}, tool: []string{"delete", "remove", "clear"}},
{name: "complete", query: []string{"完成", "提交入库", "complete", "commit"}, tool: []string{"complete", "commit", "finish"}},
{name: "enable", query: []string{"启用", "开启", "enable"}, tool: []string{"enable"}},
{name: "disable", query: []string{"禁用", "停用", "关闭", "disable"}, tool: []string{"disable"}},
}
var toolSearchProductClasses = []toolSearchTermClass{
{name: "chat", query: []string{"群聊", "群消息", "会话", "单聊", "chat"}, tool: []string{"chat"}},
{name: "calendar", query: []string{"日程", "会议日历", "会议室", "calendar"}, tool: []string{"calendar"}},
{name: "drive", query: []string{"钉盘", "云盘", "drive"}, tool: []string{"drive"}},
{name: "wiki", query: []string{"知识库", "wiki"}, tool: []string{"wiki"}},
{name: "todo", query: []string{"待办", "todo"}, tool: []string{"todo"}},
{name: "minutes", query: []string{"听记", "会议纪要", "minutes"}, tool: []string{"minutes"}},
{name: "oa", query: []string{"审批", "oa", "approval"}, tool: []string{"oa"}},
{name: "mail", query: []string{"邮件", "邮箱", "草稿", "mail", "email"}, tool: []string{"mail"}},
{name: "sheet", query: []string{"电子表格", "工作表", "单元格", "sheet", "spreadsheet"}, tool: []string{"sheet"}},
{name: "doc", query: []string{"在线文字文档", "文档正文", "adoc", "doc", "document"}, tool: []string{"doc"}},
}
var toolSearchEntityClasses = []toolSearchTermClass{
{name: "task_id", query: []string{"任务 id", "任务id", "task id", "taskid"}, tool: []string{"任务 id", "任务id", "task id", "taskid"}},
{name: "task", query: []string{"任务", "task id", "taskid", "task"}, tool: []string{"任务", "task"}},
{name: "permission", query: []string{"权限", "授权", "协作者", "permission"}, tool: []string{"权限", "授权", "协作者", "permission"}},
{name: "content", query: []string{"正文", "文档内容", "content"}, tool: []string{"正文", "内容", "content"}},
{name: "status", query: []string{"状态", "已读", "status"}, tool: []string{"状态", "已读", "status"}},
{name: "draft", query: []string{"草稿", "draft"}, tool: []string{"草稿", "draft"}},
{name: "event", query: []string{"日程", "event"}, tool: []string{"日程", "event"}},
{name: "reminder", query: []string{"提醒", "reminder"}, tool: []string{"提醒", "reminder"}},
{name: "summary", query: []string{"摘要", "summary"}, tool: []string{"摘要", "summary"}},
{name: "sheet", query: []string{"工作表", "worksheet"}, tool: []string{"工作表", "sheet"}},
{name: "range", query: []string{"单元格范围", "区域", "range"}, tool: []string{"范围", "区域", "range"}},
{name: "group", query: []string{"群聊", "群名", "group"}, tool: []string{"群聊", "群", "group"}},
}
func classifyToolSearchQuery(query string) toolSearchQueryClass {
normalized := strings.ToLower(strings.TrimSpace(query))
result := toolSearchQueryClass{actions: map[string]bool{}, products: map[string]bool{}, entities: map[string]bool{}}
for _, class := range toolSearchActionClasses {
if containsAnyToolSearchPhrase(normalized, class.query) {
result.actions[class.name] = true
}
}
for _, class := range toolSearchProductClasses {
if containsAnyToolSearchPhrase(normalized, class.query) {
result.products[class.name] = true
}
}
for _, class := range toolSearchEntityClasses {
if class.name == "task" && result.entities["task_id"] {
continue
}
if containsAnyToolSearchPhrase(normalized, class.query) {
result.entities[class.name] = true
}
}
// “查询” is intentionally only a broad read-effect signal. Treating it as
// get/list/search would impose a false sibling preference.
if strings.Contains(normalized, "查询") || result.actions["read"] || result.actions["search"] || result.actions["list"] {
result.effect = "read"
}
return result
}
func toolSearchStructuredMultiplier(query toolSearchQueryClass, tool ToolSpec) float64 {
multiplier := 1.0
if query.effect != "" {
if tool.Safety.Effect == query.effect {
multiplier += 0.10
} else {
multiplier -= 0.50
}
}
if len(query.products) > 0 {
if query.products[tool.Identity.ProductID] {
multiplier += 0.50
} else {
multiplier -= 0.30
}
}
toolText := strings.ToLower(strings.Join([]string{
tool.Identity.Name,
tool.Identity.CLIName,
tool.Identity.CanonicalPath,
tool.Identity.PrimaryCLIPath,
tool.Title,
tool.Selection.AgentSummary,
}, " "))
if len(query.actions) > 0 {
matched := false
for _, class := range toolSearchActionClasses {
if query.actions[class.name] && containsAnyToolSearchWord(toolText, class.tool) {
matched = true
break
}
}
if matched {
multiplier += 0.60
} else {
// An explicit action is a stronger constraint than generic lexical
// overlap. Penalize tools that do not advertise that action even
// when their sibling verb is not in the compact action vocabulary.
multiplier -= 0.65
}
}
if len(query.entities) > 0 {
matched := false
for _, class := range toolSearchEntityClasses {
if query.entities[class.name] && containsAnyToolSearchPhrase(toolText, class.tool) {
matched = true
break
}
}
if matched {
multiplier += 0.35
} else {
multiplier -= 0.10
}
}
if multiplier < 0.20 {
return 0.20
}
return multiplier
}
func containsAnyToolSearchPhrase(text string, values []string) bool {
for _, value := range values {
if toolSearchPhraseContains(text, value) {
return true
}
}
return false
}
// toolSearchPhraseContains matches ASCII single-token values on word
// boundaries so short English values such as "oa" cannot fire inside
// unrelated words ("download" contains "oa"). Mixed-script or multi-word
// values keep substring semantics, which is what Chinese phrases and values
// like "task id" rely on.
func toolSearchPhraseContains(text, value string) bool {
if value != "" && isAllToolSearchASCII(value) && !strings.ContainsAny(value, " \t") {
return containsToolSearchASCIIWord(text, value)
}
return strings.Contains(text, value)
}
func isAllToolSearchASCII(value string) bool {
for _, character := range value {
if character >= utf8.RuneSelf {
return false
}
}
return true
}
func containsToolSearchASCIIWord(text, word string) bool {
word = strings.ToLower(word)
for _, token := range toolSearchASCIIWordPattern.FindAllString(strings.ToLower(text), -1) {
if token == word {
return true
}
}
return false
}
func containsAnyToolSearchWord(text string, values []string) bool {
normalized := strings.NewReplacer(".", " ", "_", " ", "-", " ", "/", " ").Replace(text)
words := strings.Fields(normalized)
for _, value := range values {
for _, word := range words {
if word == value {
return true
}
}
}
return false
}
func newToolSearchExplainQueryClass(query toolSearchQueryClass) *ToolSearchExplainQueryClass {
return &ToolSearchExplainQueryClass{
Actions: sortedToolSearchClassKeys(query.actions),
Products: sortedToolSearchClassKeys(query.products),
Entities: sortedToolSearchClassKeys(query.entities),
Effect: query.effect,
}
}
func sortedToolSearchClassKeys(set map[string]bool) []string {
if len(set) == 0 {
return nil
}
keys := make([]string, 0, len(set))
for key := range set {
keys = append(keys, key)
}
sort.Strings(keys)
return keys
}
// toolSearchAvoidWhenPenalty is applied when a candidate's avoid_when prose
// matches the query intent. It is a soft demotion, not an eligibility filter:
// avoid_when is natural-language guidance (often "prefer X for that intent"),
// so a lexical hit can be wrong and the tool must remain reachable.
const toolSearchAvoidWhenPenalty = 0.20
// toolSearchAvoidWhenShortcutNoise filters the boilerplate avoid_when that
// every shortcut carries; it carries no intent signal. Kept lowercase because
// phrases are compared in the lowercased domain.
const toolSearchAvoidWhenShortcutNoise = "需要该 shortcut 未公开的底层参数"
// toolSearchAvoidWhenPenaltyReason returns the matched avoid_when phrase, or
// "" when the tool should not be demoted for this query. The only match shape
// is the query containing the avoidance prose itself (the Forbidden@5
// evaluation proxy). Intent-overlap matching was tried and rejected: a tool's
// own avoid_when describes the same sibling scenario space as its use_when,
// so "shares an action/entity with the phrase" demotes the gold answer.
func toolSearchAvoidWhenPenaltyReason(query string, tool ToolSpec) string {
if len(tool.Selection.AvoidWhen) == 0 {
return ""
}
normalized := strings.ToLower(strings.TrimSpace(query))
if normalized == "" {
return ""
}
for _, phrase := range tool.Selection.AvoidWhen {
phrase = strings.ToLower(strings.TrimSpace(phrase))
if phrase == "" || strings.Contains(phrase, toolSearchAvoidWhenShortcutNoise) {
continue
}
if strings.Contains(normalized, phrase) {
return phrase
}
}
return ""
}
// toolSearchAvoidWhenRetriever demotes candidates whose avoid_when prose
// matches the query intent. It wraps every lexical algorithm, including the
// production ensemble default, so the Forbidden exposure guard is not tied to
// action_v1.
type toolSearchAvoidWhenRetriever struct {
base LexicalRetriever
documents map[string]toolSearchDocument
explain bool
}
func newToolSearchAvoidWhenRetriever(base LexicalRetriever, documents map[string]toolSearchDocument, explain bool) *toolSearchAvoidWhenRetriever {
return &toolSearchAvoidWhenRetriever{base: base, documents: documents, explain: explain}
}
func (r *toolSearchAvoidWhenRetriever) Name() string { return r.base.Name() }
func (r *toolSearchAvoidWhenRetriever) Retrieve(ctx context.Context, request ToolSearchLexicalRequest) ([]LexicalHit, error) {
hits, err := r.base.Retrieve(ctx, request)
if err != nil {
return nil, err
}
penalized := false
for index := range hits {
reason := toolSearchAvoidWhenPenaltyReason(request.Query, r.documents[hits[index].CanonicalPath].tool)
if reason == "" {
continue
}
hits[index].Score *= toolSearchAvoidWhenPenalty
if r.explain && hits[index].Explain != nil {
hits[index].Explain.Score = hits[index].Score
hits[index].Explain.AvoidWhenPenalty = reason
}
penalized = true
}
if !penalized {
return hits, nil
}
return truncateAndSortLexicalHits(hits, request.CandidateLimit), nil
}
type toolSearchBM25Retriever struct {
fieldIndex map[toolSearchField]toolSearchBM25Index
weights ToolSearchFieldWeights
explain bool
}
func newToolSearchBM25Retriever(documents map[string]toolSearchDocument, weights ToolSearchFieldWeights, k1, b float64, explain bool) *toolSearchBM25Retriever {
fieldTerms := toolSearchFieldTerms(documents)
fieldIndex := make(map[toolSearchField]toolSearchBM25Index, len(toolSearchFieldOrder))
for _, field := range toolSearchFieldOrder {
fieldIndex[field] = newToolSearchBM25Index(fieldTerms[field], k1, b)
}
return &toolSearchBM25Retriever{fieldIndex: fieldIndex, weights: weights, explain: explain}
}
func (*toolSearchBM25Retriever) Name() string { return ToolSearchLexicalBM25 }
func (r *toolSearchBM25Retriever) Retrieve(ctx context.Context, request ToolSearchLexicalRequest) ([]LexicalHit, error) {
queryTerms := orderedToolSearchQueryTerms(request.Query)
hits := make([]LexicalHit, 0, len(request.EligibleCanonicalPaths))
for _, canonical := range request.EligibleCanonicalPaths {
if err := ctx.Err(); err != nil {
return nil, err
}
contributions := make(map[toolSearchField]float64)
score := 0.0
for _, field := range toolSearchFieldOrder {
weight := toolSearchFieldWeight(r.weights, field)
if weight <= 0 {
continue
}
value := r.fieldIndex[field].score(canonical, queryTerms) * weight
if value > 0 {
contributions[field] = value
score += value
}
}
if score > 0 {
hit := LexicalHit{CanonicalPath: canonical, Score: score, MatchedFields: matchedToolSearchFields(contributions)}
if r.explain {
hit.Explain = newToolSearchBreakdown(score, contributions)
}
hits = append(hits, hit)
}
}
return truncateAndSortLexicalHits(hits, request.CandidateLimit), nil
}
type toolSearchTFIDFDocument struct {
weights map[string]float64
norm float64
}
type toolSearchTFIDFIndex struct {
documents map[string]toolSearchTFIDFDocument
idf map[string]float64
}
type toolSearchTFIDFRetriever struct {
fieldIndex map[toolSearchField]toolSearchTFIDFIndex
weights ToolSearchFieldWeights
explain bool
}
func newToolSearchTFIDFRetriever(documents map[string]toolSearchDocument, weights ToolSearchFieldWeights, explain bool) *toolSearchTFIDFRetriever {
fieldTerms := toolSearchFieldTerms(documents)
fieldIndex := make(map[toolSearchField]toolSearchTFIDFIndex, len(toolSearchFieldOrder))
for _, field := range toolSearchFieldOrder {
fieldIndex[field] = newToolSearchTFIDFIndex(fieldTerms[field])
}
return &toolSearchTFIDFRetriever{fieldIndex: fieldIndex, weights: weights, explain: explain}
}
func (*toolSearchTFIDFRetriever) Name() string { return ToolSearchLexicalTFIDF }
func (r *toolSearchTFIDFRetriever) Retrieve(ctx context.Context, request ToolSearchLexicalRequest) ([]LexicalHit, error) {
queryFrequency := toolSearchTermFrequency(tokenizeToolSearchText(request.Query))
hits := make([]LexicalHit, 0, len(request.EligibleCanonicalPaths))
for _, canonical := range request.EligibleCanonicalPaths {
if err := ctx.Err(); err != nil {
return nil, err
}
contributions := make(map[toolSearchField]float64)
score := 0.0
for _, field := range toolSearchFieldOrder {
weight := toolSearchFieldWeight(r.weights, field)
if weight <= 0 {
continue
}
value := r.fieldIndex[field].score(canonical, queryFrequency) * weight
if value > 0 {
contributions[field] = value
score += value
}
}
if score > 0 {
hit := LexicalHit{CanonicalPath: canonical, Score: score, MatchedFields: matchedToolSearchFields(contributions)}
if r.explain {
hit.Explain = newToolSearchBreakdown(score, contributions)
}
hits = append(hits, hit)
}
}
return truncateAndSortLexicalHits(hits, request.CandidateLimit), nil
}
func toolSearchFieldTerms(documents map[string]toolSearchDocument) map[toolSearchField]map[string][]string {
fieldTerms := make(map[toolSearchField]map[string][]string, len(toolSearchFieldOrder))
for _, field := range toolSearchFieldOrder {
fieldTerms[field] = make(map[string][]string, len(documents))
}
for canonical, document := range documents {
for _, field := range toolSearchFieldOrder {
fieldTerms[field][canonical] = tokenizeToolSearchText(document.fields[field])
}
}
return fieldTerms
}
func orderedToolSearchQueryTerms(query string) []toolSearchQueryTerm {
frequency := toolSearchTermFrequency(tokenizeToolSearchText(query))
terms := make([]toolSearchQueryTerm, 0, len(frequency))
for term, count := range frequency {
terms = append(terms, toolSearchQueryTerm{term: term, count: count})
}
sort.Slice(terms, func(i, j int) bool { return terms[i].term < terms[j].term })
return terms
}
func toolSearchTermFrequency(tokens []string) map[string]int {
frequency := make(map[string]int, len(tokens))
for _, token := range tokens {
frequency[token]++
}
return frequency
}
func newToolSearchTFIDFIndex(documents map[string][]string) toolSearchTFIDFIndex {
index := toolSearchTFIDFIndex{documents: make(map[string]toolSearchTFIDFDocument, len(documents)), idf: map[string]float64{}}
documentFrequency := map[string]int{}
frequencies := make(map[string]map[string]int, len(documents))
for canonical, tokens := range documents {
frequency := toolSearchTermFrequency(tokens)
frequencies[canonical] = frequency
for term := range frequency {
documentFrequency[term]++
}
}
n := float64(len(documents))
for term, frequency := range documentFrequency {
index.idf[term] = math.Log((n+1)/(float64(frequency)+1)) + 1
}
for canonical, frequency := range frequencies {
weights := make(map[string]float64, len(frequency))
normSquared := 0.0
terms := make([]string, 0, len(frequency))
for term := range frequency {
terms = append(terms, term)
}
sort.Strings(terms)
for _, term := range terms {
count := frequency[term]
value := float64(count) * index.idf[term]
weights[term] = value
normSquared += value * value
}
index.documents[canonical] = toolSearchTFIDFDocument{weights: weights, norm: math.Sqrt(normSquared)}
}
return index
}
func (i toolSearchTFIDFIndex) score(canonical string, queryFrequency map[string]int) float64 {
document, ok := i.documents[canonical]
if !ok || document.norm == 0 {
return 0
}
terms := make([]string, 0, len(queryFrequency))
for term := range queryFrequency {
terms = append(terms, term)
}
sort.Strings(terms)
dot := 0.0
queryNormSquared := 0.0
for _, term := range terms {
idf, ok := i.idf[term]
if !ok {
continue
}
queryWeight := float64(queryFrequency[term]) * idf
queryNormSquared += queryWeight * queryWeight
dot += document.weights[term] * queryWeight
}
if queryNormSquared == 0 {
return 0
}
return dot / (document.norm * math.Sqrt(queryNormSquared))
}
// toolSearchScoreQuantum is the sort-key quantization grid. Go permits FMA
// fusion on arm64 but not amd64, so the same ranking expression can differ in
// the last ulp across architectures; ordering on the raw float would let that
// ulp flip near-tied candidates past the canonical tie-break and break the
// byte-determinism contract across CI (linux/amd64) and dev (macOS/arm64)
// machines. Quantizing only the ordering key keeps same-architecture output
// identical while making cross-architecture ordering agree; reported scores
// (explain-only) are untouched.
const toolSearchScoreQuantum = 1e9
func quantizeToolSearchScore(score float64) float64 {
return math.Round(score*toolSearchScoreQuantum) / toolSearchScoreQuantum
}
func truncateAndSortLexicalHits(hits []LexicalHit, limit int) []LexicalHit {
sort.Slice(hits, func(i, j int) bool {
left, right := quantizeToolSearchScore(hits[i].Score), quantizeToolSearchScore(hits[j].Score)
if left != right {
return left > right
}
return hits[i].CanonicalPath < hits[j].CanonicalPath
})
if limit >= 0 && len(hits) > limit {
hits = hits[:limit]
}
return hits
}
File diff suppressed because it is too large Load Diff
+8 -3
View File
@@ -361,13 +361,18 @@ func TestCrossPlatformCoverageDaemonMethodEdges(t *testing.T) {
}
return nil, net.ErrClosed
}}
d := &daemon{listener: l, log: logger, hub: NewHub(1), idleStop: make(chan struct{})}
d := &daemon{listener: l, log: logger, hub: NewHub(1), idleStop: make(chan struct{}), stopReq: make(chan string, 1)}
d.acceptLoop(context.Background())
d.shuttingDown.Store(true)
d.acceptLoop(context.Background())
d.triggerShutdown("test")
if !l.closed {
t.Fatal("trigger did not close listener")
select {
case reason := <-d.stopReq:
if reason != "test" {
t.Fatalf("stop reason = %q", reason)
}
default:
t.Fatal("trigger did not notify lifecycle loop")
}
d = &daemon{listener: &scriptedListener{accept: func() (net.Conn, error) { return nil, net.ErrClosed }}, log: logger, hub: NewHub(1), idleStop: make(chan struct{})}
+14 -7
View File
@@ -186,6 +186,7 @@ func Run(ctx context.Context, cfg Config) error {
dedup: dd,
started: time.Now().UTC(),
idleStop: make(chan struct{}),
stopReq: make(chan string, 1),
}
// 4. Signal ready BEFORE accepting consumers (avoids a slow-fork
@@ -263,6 +264,9 @@ func Run(ctx context.Context, cfg Config) error {
}
case <-d.idleStop:
log.Info("bus: idle timeout reached, shutting down")
case reason := <-d.stopReq:
shutdownReason = reason
log.Info("bus: shutdown requested via IPC", "reason", reason)
}
// 7. Graceful shutdown — cancel runCtx first so all background
@@ -292,6 +296,7 @@ type daemon struct {
shutdownMu sync.Mutex
shuttingDown atomic.Bool
idleStop chan struct{}
stopReq chan string
credentialHandoffMu sync.Mutex
terminalMu sync.RWMutex
@@ -749,15 +754,17 @@ func (d *daemon) idleWatch(ctx context.Context) {
}
}
// triggerShutdown is called from RPC handlers that want to end the bus.
// It works by closing the listener (which unblocks Run's select via the
// source error path, indirectly). For v1 a full ctx-cancellation hook is
// out of scope; busctl/stop also sends SIGTERM which is the authoritative
// shutdown path.
// triggerShutdown is called from RPC handlers that want to end the bus. The
// buffered request wakes Run's lifecycle select, which then cancels the cloud
// source and performs the same graceful shutdown used for signals and idle
// expiry. It is intentionally non-blocking so repeated stop requests cannot
// strand connection handlers.
func (d *daemon) triggerShutdown(reason string) {
d.log.Info("bus: shutdown triggered via IPC", "reason", reason)
_ = d.listener.Close() // unblocks Accept(), but doesn't kill Source
// Best-effort: a future version wires a context.CancelFunc here.
select {
case d.stopReq <- reason:
default:
}
}
// shutdown performs the graceful tear-down sequence:
+61
View File
@@ -141,6 +141,67 @@ func TestDaemon_RunStartsAndShutsDownCleanly(t *testing.T) {
}
}
func TestCrossPlatformCoverageDaemonIPCStopShutsDownCleanly(t *testing.T) {
workDir := shortTempDir(t)
identityHash := dwsevent.IdentityHash(workDir)
endpoint := dwsevent.IPCEndpoint(workDir, "open", dwsevent.SourceKindAppStream, identityHash)
runDone := make(chan error, 1)
go func() {
runDone <- Run(context.Background(), Config{
WorkDir: workDir,
IPCEndpoint: endpoint,
ClientID: "ding_ipc_stop_test",
Edition: "open",
SourceKind: dwsevent.SourceKindAppStream,
IdentityHash: identityHash,
Source: &fakeSource{},
})
}()
var conn net.Conn
deadline := time.Now().Add(3 * time.Second)
for time.Now().Before(deadline) {
var err error
conn, err = transport.Dial(endpoint)
if err == nil {
break
}
time.Sleep(10 * time.Millisecond)
}
if conn == nil {
t.Fatal("bus IPC endpoint did not become ready")
}
w := transport.NewWriter(conn)
r := transport.NewReader(conn)
if err := w.WriteJSON(transport.Hello{
Type: transport.FrameTypeHello,
ConsumerPID: os.Getpid(),
Role: transport.HelloRoleStop,
}); err != nil {
t.Fatalf("write stop hello: %v", err)
}
var bye transport.Bye
if err := r.ReadJSON(&bye); err != nil {
t.Fatalf("read stop response: %v", err)
}
_ = conn.Close()
if bye.Type != transport.FrameTypeBye || bye.Reason != "stop_request" {
t.Fatalf("stop response = %#v", bye)
}
select {
case err := <-runDone:
if err != nil {
t.Fatalf("Run returned after IPC stop: %v", err)
}
case <-time.After(3 * time.Second):
t.Fatal("Run did not return after IPC stop")
}
if pid := ReadHolderPID(filepath.Join(workDir, LockFileName)); pid != 0 {
t.Fatalf("bus lock retained pid %d after IPC stop", pid)
}
}
func TestDaemon_ConsumerReceivesEvents(t *testing.T) {
skipOnWindows(t, "uses Unix socket dial")
workDir := shortTempDir(t)
+41
View File
@@ -114,6 +114,47 @@ func ReadHolderPID(path string) int {
return parsePID(b)
}
// ValidateHolderOwnership reports whether pid still owns the exclusive lock at
// path. If the lock can be acquired, no daemon owns it: the PID payload is
// stale and is cleared while the probe holds the lock. The PID is read again
// after a busy result so a daemon replacement cannot be mistaken for the
// original owner.
//
// Callers use this immediately before sending a process-level termination
// fallback. A live PID by itself is not proof of ownership because operating
// systems may reuse PIDs after an unclean daemon exit.
func ValidateHolderOwnership(path string, pid int) (bool, error) {
if pid <= 0 || ReadHolderPID(path) != pid {
return false, nil
}
probe, err := busTryAcquire(path)
if err != nil {
if errors.Is(err, ErrBusy) {
return ReadHolderPID(path) == pid, nil
}
return false, fmt.Errorf("bus: verify lock owner: %w", err)
}
defer probe.Close()
// We own the lock, so the recorded PID cannot own it. Clear only the PID
// we validated above; a changed payload is left untouched defensively.
f := probe.File()
if _, err := busSeek(f, 0, io.SeekStart); err != nil {
return false, fmt.Errorf("bus: seek stale lock: %w", err)
}
current, err := busReadAll(f)
if err != nil {
return false, fmt.Errorf("bus: read stale lock: %w", err)
}
if parsePID(current) == pid {
if err := truncateAndWritePID(f, 0); err != nil {
return false, fmt.Errorf("bus: clear stale PID: %w", err)
}
}
return false, nil
}
// Close releases the flock and best-effort blanks the PID body so a stale
// reader (e.g. `event status` racing our shutdown) does not see our
// long-dead PID and try to signal it. The lock file itself is NOT removed
+107
View File
@@ -16,9 +16,13 @@ package bus
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"testing"
eventlock "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/lock"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
)
func TestAcquire_WritesOurPID(t *testing.T) {
@@ -112,6 +116,109 @@ func TestReadHolderPID_MalformedReturnsZero(t *testing.T) {
}
}
func TestCrossPlatformCoverageValidateHolderOwnershipHeldLock(t *testing.T) {
path := filepath.Join(t.TempDir(), LockFileName)
held, err := busTryAcquire(path)
if err != nil {
t.Fatalf("hold lock: %v", err)
}
defer held.Close()
if err := truncateAndWritePID(held.File(), os.Getpid()); err != nil {
t.Fatalf("write holder PID: %v", err)
}
owned, err := ValidateHolderOwnership(path, os.Getpid())
if err != nil {
t.Fatalf("ValidateHolderOwnership: %v", err)
}
if !owned {
t.Fatal("held lock was not attributed to its recorded PID")
}
}
func TestCrossPlatformCoverageValidateHolderOwnershipClearsReusedPID(t *testing.T) {
path := filepath.Join(t.TempDir(), LockFileName)
// The current PID is alive but deliberately does not hold this lock. This
// models a stale daemon PID that the OS has reassigned to another process.
if err := os.WriteFile(path, []byte(fmt.Sprintf("%d\n", os.Getpid())), 0o600); err != nil {
t.Fatalf("write stale PID: %v", err)
}
owned, err := ValidateHolderOwnership(path, os.Getpid())
if err != nil {
t.Fatalf("ValidateHolderOwnership: %v", err)
}
if owned {
t.Fatal("live reused PID without the bus lock was accepted as owner")
}
if got := ReadHolderPID(path); got != 0 {
t.Fatalf("stale PID after validation = %d, want cleared", got)
}
}
func TestCrossPlatformCoverageValidateHolderOwnershipRejectsMismatch(t *testing.T) {
path := filepath.Join(t.TempDir(), LockFileName)
if err := os.WriteFile(path, []byte("123\n"), 0o600); err != nil {
t.Fatal(err)
}
owned, err := ValidateHolderOwnership(path, 456)
if err != nil || owned {
t.Fatalf("ValidateHolderOwnership mismatch = %v, %v", owned, err)
}
}
func TestCrossPlatformCoverageValidateHolderOwnershipErrors(t *testing.T) {
errInjected := errors.New("injected ownership validation failure")
writeLivePID := func(t *testing.T) string {
t.Helper()
path := filepath.Join(t.TempDir(), LockFileName)
if err := os.WriteFile(path, []byte(fmt.Sprintf("%d\n", os.Getpid())), 0o600); err != nil {
t.Fatal(err)
}
return path
}
t.Run("probe", func(t *testing.T) {
path := writeLivePID(t)
testseam.Swap(t, &busTryAcquire, func(string) (*eventlock.File, error) {
return nil, errInjected
})
if _, err := ValidateHolderOwnership(path, os.Getpid()); !errors.Is(err, errInjected) {
t.Fatalf("probe error = %v", err)
}
})
t.Run("seek", func(t *testing.T) {
path := writeLivePID(t)
testseam.Swap(t, &busSeek, func(*os.File, int64, int) (int64, error) {
return 0, errInjected
})
if _, err := ValidateHolderOwnership(path, os.Getpid()); !errors.Is(err, errInjected) {
t.Fatalf("seek error = %v", err)
}
})
t.Run("read", func(t *testing.T) {
path := writeLivePID(t)
testseam.Swap(t, &busReadAll, func(io.Reader) ([]byte, error) {
return nil, errInjected
})
if _, err := ValidateHolderOwnership(path, os.Getpid()); !errors.Is(err, errInjected) {
t.Fatalf("read error = %v", err)
}
})
t.Run("clear", func(t *testing.T) {
path := writeLivePID(t)
testseam.Swap(t, &busTruncate, func(*os.File, int64) error {
return errInjected
})
if _, err := ValidateHolderOwnership(path, os.Getpid()); !errors.Is(err, errInjected) {
t.Fatalf("clear error = %v", err)
}
})
}
func TestAcquire_AfterReleaseReclaimable(t *testing.T) {
path := filepath.Join(t.TempDir(), LockFileName)
l1, err := Acquire(path)
@@ -284,11 +284,13 @@ func TestCrossPlatformCoverageQueryStatusAndEntryEdges(t *testing.T) {
func TestCrossPlatformCoverageStopInjectedEdges(t *testing.T) {
origRead := stopReadHolderPID
origValidateOwner := stopValidateHolderOwner
origAlive := stopAlive
origFind := stopFindProcess
origSignal := stopSignalProcess
t.Cleanup(func() {
stopReadHolderPID = origRead
stopValidateHolderOwner = origValidateOwner
stopAlive = origAlive
stopFindProcess = origFind
stopSignalProcess = origSignal
@@ -307,6 +309,7 @@ func TestCrossPlatformCoverageStopInjectedEdges(t *testing.T) {
t.Fatal(err)
}
stopFindProcess = func(int) (*os.Process, error) { return proc, nil }
stopValidateHolderOwner = func(string, int) (bool, error) { return true, nil }
stopSignalProcess = func(*os.Process, os.Signal) error { return os.ErrProcessDone }
if err := Stop(StopConfig{WorkDir: "x"}); err != nil {
t.Fatalf("done signal = %v", err)
+3 -2
View File
@@ -112,8 +112,9 @@ func Discover(cfg DiscoverConfig) (net.Conn, error) {
return nil, fmt.Errorf("busctl: mkdir workdir: %w", err)
}
_, spawnErr := cfg.Spawn(SpawnConfig{
ClientID: cfg.ClientID,
ExtraArgs: cfg.SpawnExtraArgs,
ClientID: cfg.ClientID,
IPCEndpoint: cfg.IPCEndpoint,
ExtraArgs: cfg.SpawnExtraArgs,
})
if spawnErr != nil && !errors.Is(spawnErr, ErrSpawnFailed) {
// Hard error (couldn't even exec the child). Stop here — no bus
+4 -1
View File
@@ -122,7 +122,10 @@ func TestDiscover_NoBus_SpawnSucceeds(t *testing.T) {
closer()
}
})
fakeSpawn := func(SpawnConfig) (int, error) {
fakeSpawn := func(cfg SpawnConfig) (int, error) {
if cfg.IPCEndpoint != sock {
t.Fatalf("spawn IPC endpoint = %q, want %q", cfg.IPCEndpoint, sock)
}
closer = startStubBus(t, sock)
return 12345, nil
}
+15 -14
View File
@@ -43,18 +43,14 @@ var (
spawnPipe = os.Pipe
)
// ErrSpawnFailed is returned when the child reports startup failure via
// the ready pipe ('E' byte). The child's exit error / log file holds the
// actual cause; this sentinel just lets the caller distinguish "ready
// pipe said no" from "ready pipe timed out / closed early".
var ErrSpawnFailed = errors.New("busctl: bus child reported startup failure on ready pipe")
// ErrSpawnFailed is returned when the child reports startup failure through
// the Unix ready pipe or exits before binding the Windows named pipe.
var ErrSpawnFailed = errors.New("busctl: bus child reported startup failure")
// ErrSpawnTimeout is returned when ReadyTimeout elapses without any signal.
var ErrSpawnTimeout = errors.New("busctl: bus child did not signal readiness within deadline")
// SpawnConfig describes one spawn attempt. ClientID is the only field
// inspected by the child; the rest govern process attributes the parent
// applies before exec.
// SpawnConfig describes one spawn attempt.
type SpawnConfig struct {
// ExecPath is the dws binary to exec. Default os.Executable().
ExecPath string
@@ -62,18 +58,23 @@ type SpawnConfig struct {
// ClientID is passed as `--client-id` to `dws event _bus`. Required.
ClientID string
// IPCEndpoint is the bus endpoint the child binds. Windows uses the
// existing named pipe as its readiness handshake because os/exec does not
// support ExtraFiles there. Unix keeps the inherited ready-pipe protocol.
IPCEndpoint string
// ExtraArgs are appended after `--client-id`. Empty for normal use; tests
// pass `--extra-flag-for-test` etc.
ExtraArgs []string
// Env to pass to the child. Defaults to os.Environ(). The ReadyFDEnv
// entry is appended automatically.
// Env to pass to the child. Defaults to os.Environ(). Unix appends the
// ReadyFDEnv entry automatically; Windows removes any inherited copy.
Env []string
}
// Spawn forks a detached `dws event _bus --client-id <id>` child process and
// waits for it to signal readiness via the ready pipe. Returns the child's
// PID on success — the caller can then dial the bus IPC endpoint.
// spawnWithReadyPipe is the Unix implementation behind Spawn. It forks a
// detached `dws event _bus --client-id <id>` child and waits for the inherited
// ready pipe. Windows has a named-pipe implementation in spawn_windows.go.
//
// stdio detach (plan invariant #7):
// - cmd.Stdout / cmd.Stderr set to nil so the child's own writes don't
@@ -89,7 +90,7 @@ type SpawnConfig struct {
// Parent (this function):
// - Holds the read end open until either 1 byte is read or ReadyTimeout
// - Returns ErrSpawnFailed for 'E', ErrSpawnTimeout otherwise
func Spawn(cfg SpawnConfig) (pid int, err error) {
func spawnWithReadyPipe(cfg SpawnConfig) (pid int, err error) {
if cfg.ClientID == "" {
return 0, errors.New("busctl: SpawnConfig.ClientID is required")
}
+24 -1
View File
@@ -22,6 +22,8 @@ import (
"strings"
"testing"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/transport"
)
// Test child mode: when DWS_BUSCTL_TEST_CHILD is set, this test binary
@@ -35,7 +37,8 @@ import (
// env-marker pattern is what Go's own os/exec tests use and stays
// confined to this file.
const (
childEnvMarker = "DWS_BUSCTL_TEST_CHILD"
childEnvMarker = "DWS_BUSCTL_TEST_CHILD"
childEndpointEnv = "DWS_BUSCTL_TEST_ENDPOINT"
// values:
// "ready" — write 'R' then sleep 30s (parent should see ready)
// "fail" — write 'E' then exit (parent should see ErrSpawnFailed)
@@ -70,6 +73,26 @@ func TestMain(m *testing.M) {
writeReady('R')
time.Sleep(30 * time.Second)
os.Exit(0)
case "windows-ready":
listener, err := transport.Listen(os.Getenv(childEndpointEnv))
if err != nil {
os.Exit(3)
}
defer listener.Close()
conn, err := listener.Accept()
if err != nil {
os.Exit(4)
}
_ = conn.Close()
time.Sleep(30 * time.Second)
os.Exit(0)
case "windows-fail":
os.Exit(5)
case "windows-exit":
os.Exit(0)
case "windows-stall":
time.Sleep(30 * time.Second)
os.Exit(0)
}
os.Exit(m.Run())
}
+7
View File
@@ -20,6 +20,13 @@ import (
"syscall"
)
// Spawn starts the detached bus and waits for its inherited ready pipe.
// ExtraFiles is intentionally confined to Unix: Go does not support it on
// Windows.
func Spawn(cfg SpawnConfig) (pid int, err error) {
return spawnWithReadyPipe(cfg)
}
// applyDetach configures the child to live past parent death and not share
// the parent's controlling terminal. Setsid puts the child in a new
// session, so SIGHUP on the parent's controlling tty (e.g. SSH disconnect)
+87
View File
@@ -16,14 +16,101 @@
package busctl
import (
"fmt"
"os"
"os/exec"
"strings"
"syscall"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/transport"
)
// CREATE_NEW_PROCESS_GROUP (0x00000200) prevents the child from receiving
// the parent's Ctrl+C signal, similar in spirit to Setsid on Unix.
const createNewProcessGroup = 0x00000200
var (
spawnWindowsDial = transport.Dial
spawnWindowsPollInterval = 25 * time.Millisecond
)
// Spawn starts a detached Windows bus without cmd.ExtraFiles (unsupported by
// Go on Windows). Readiness is confirmed by dialing the bus's existing named
// pipe. The child is reaped in the background after readiness, while an early
// process exit is surfaced as ErrSpawnFailed.
func Spawn(cfg SpawnConfig) (pid int, err error) {
if strings.TrimSpace(cfg.ClientID) == "" {
return 0, fmt.Errorf("busctl: SpawnConfig.ClientID is required")
}
if strings.TrimSpace(cfg.IPCEndpoint) == "" {
return 0, fmt.Errorf("busctl: SpawnConfig.IPCEndpoint is required on Windows")
}
if cfg.ExecPath == "" {
execPath, err := spawnExecutable()
if err != nil {
return 0, fmt.Errorf("busctl: locate executable: %w", err)
}
cfg.ExecPath = execPath
}
if cfg.Env == nil {
cfg.Env = os.Environ()
}
args := append([]string{"event", "_bus", "--client-id", cfg.ClientID}, cfg.ExtraArgs...)
cmd := exec.Command(cfg.ExecPath, args...)
cmd.Env = withoutReadyFDEnv(cfg.Env)
cmd.Stdin = nil
cmd.Stdout = nil
cmd.Stderr = nil
applyDetach(cmd)
if err := cmd.Start(); err != nil {
return 0, fmt.Errorf("busctl: start %s: %w", cfg.ExecPath, err)
}
pid = cmd.Process.Pid
waitDone := make(chan error, 1)
go func() { waitDone <- cmd.Wait() }()
timer := time.NewTimer(ReadyTimeout)
defer timer.Stop()
ticker := time.NewTicker(spawnWindowsPollInterval)
defer ticker.Stop()
for {
if conn, dialErr := spawnWindowsDial(cfg.IPCEndpoint); dialErr == nil {
_ = conn.Close()
return pid, nil
}
select {
case waitErr := <-waitDone:
if waitErr == nil {
return pid, fmt.Errorf("%w: child exited before binding named pipe", ErrSpawnFailed)
}
return pid, fmt.Errorf("%w: %v", ErrSpawnFailed, waitErr)
case <-timer.C:
_ = cmd.Process.Kill()
select {
case <-waitDone:
case <-time.After(time.Second):
}
return pid, ErrSpawnTimeout
case <-ticker.C:
}
}
}
func withoutReadyFDEnv(env []string) []string {
prefix := strings.ToUpper(ReadyFDEnv) + "="
out := make([]string, 0, len(env))
for _, entry := range env {
if strings.HasPrefix(strings.ToUpper(entry), prefix) {
continue
}
out = append(out, entry)
}
return out
}
func applyDetach(cmd *exec.Cmd) {
cmd.SysProcAttr = &syscall.SysProcAttr{
CreationFlags: createNewProcessGroup,
+132
View File
@@ -0,0 +1,132 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build windows
package busctl
import (
"errors"
"fmt"
"os"
"strings"
"testing"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
)
func TestCrossPlatformCoverageWindowsSpawnUsesNamedPipeReadiness(t *testing.T) {
endpoint := fmt.Sprintf(`\\.\pipe\dws-event-spawn-test-%d-%d`, os.Getpid(), time.Now().UnixNano())
pid, err := spawnWithMarker(t, "windows-ready", func(cfg *SpawnConfig) {
cfg.IPCEndpoint = endpoint
cfg.Env = append(cfg.Env, childEndpointEnv+"="+endpoint)
})
if err != nil {
t.Fatalf("Spawn() = %v", err)
}
if pid <= 0 {
t.Fatalf("Spawn() pid = %d", pid)
}
proc, err := os.FindProcess(pid)
if err != nil {
t.Fatal(err)
}
_ = proc.Kill()
}
func TestCrossPlatformCoverageWindowsSpawnReportsEarlyExit(t *testing.T) {
endpoint := fmt.Sprintf(`\\.\pipe\dws-event-spawn-fail-%d-%d`, os.Getpid(), time.Now().UnixNano())
_, err := spawnWithMarker(t, "windows-fail", func(cfg *SpawnConfig) {
cfg.IPCEndpoint = endpoint
})
if !errors.Is(err, ErrSpawnFailed) {
t.Fatalf("Spawn() error = %v, want ErrSpawnFailed", err)
}
}
func TestCrossPlatformCoverageWindowsSpawnReportsCleanEarlyExit(t *testing.T) {
endpoint := fmt.Sprintf(`\\.\pipe\dws-event-spawn-exit-%d-%d`, os.Getpid(), time.Now().UnixNano())
_, err := spawnWithMarker(t, "windows-exit", func(cfg *SpawnConfig) {
cfg.IPCEndpoint = endpoint
})
if !errors.Is(err, ErrSpawnFailed) {
t.Fatalf("Spawn() error = %v, want ErrSpawnFailed", err)
}
}
func TestCrossPlatformCoverageWindowsSpawnResolvesExecutable(t *testing.T) {
endpoint := fmt.Sprintf(`\\.\pipe\dws-event-spawn-resolve-%d-%d`, os.Getpid(), time.Now().UnixNano())
testseam.Swap(t, &spawnExecutable, func() (string, error) { return os.Args[0], nil })
pid, err := spawnWithMarker(t, "windows-ready", func(cfg *SpawnConfig) {
cfg.ExecPath = ""
cfg.IPCEndpoint = endpoint
cfg.Env = append(cfg.Env, childEndpointEnv+"="+endpoint)
})
if err != nil {
t.Fatalf("Spawn() = %v", err)
}
proc, err := os.FindProcess(pid)
if err != nil {
t.Fatal(err)
}
_ = proc.Kill()
}
func TestCrossPlatformCoverageWindowsSpawnValidationAndStartErrors(t *testing.T) {
endpoint := fmt.Sprintf(`\\.\pipe\dws-event-spawn-errors-%d-%d`, os.Getpid(), time.Now().UnixNano())
if _, err := Spawn(SpawnConfig{ClientID: "client"}); err == nil {
t.Fatal("missing endpoint unexpectedly succeeded")
}
testseam.Swap(t, &spawnExecutable, func() (string, error) {
return "", errors.New("executable unavailable")
})
if _, err := Spawn(SpawnConfig{ClientID: "client", IPCEndpoint: endpoint}); err == nil {
t.Fatal("executable lookup unexpectedly succeeded")
}
if _, err := Spawn(SpawnConfig{
ClientID: "client",
IPCEndpoint: endpoint,
ExecPath: `C:\\definitely-missing-dws.exe`,
}); err == nil {
t.Fatal("missing executable unexpectedly started")
}
}
func TestCrossPlatformCoverageWindowsSpawnReadyTimeoutTerminatesChild(t *testing.T) {
endpoint := fmt.Sprintf(`\\.\pipe\dws-event-spawn-stall-%d-%d`, os.Getpid(), time.Now().UnixNano())
testseam.Swap(t, &ReadyTimeout, 100*time.Millisecond)
_, err := spawnWithMarker(t, "windows-stall", func(cfg *SpawnConfig) {
cfg.IPCEndpoint = endpoint
})
if !errors.Is(err, ErrSpawnTimeout) {
t.Fatalf("Spawn() error = %v, want ErrSpawnTimeout", err)
}
}
func TestCrossPlatformCoverageWindowsSpawnRemovesInheritedReadyFD(t *testing.T) {
env := withoutReadyFDEnv([]string{
"PATH=C:\\Windows",
ReadyFDEnv + "=3",
strings.ToLower(ReadyFDEnv) + "=4",
})
if len(env) != 1 || !strings.HasPrefix(env[0], "PATH=") {
t.Fatalf("withoutReadyFDEnv() = %#v", env)
}
}
func TestCrossPlatformCoverageWindowsUnixReadyPipeValidation(t *testing.T) {
if _, err := spawnWithReadyPipe(SpawnConfig{}); err == nil {
t.Fatal("spawnWithReadyPipe() unexpectedly accepted an empty client ID")
}
}
+95 -24
View File
@@ -17,15 +17,17 @@ import (
"errors"
"fmt"
"os"
"strings"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/bus"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/process"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/transport"
)
// DefaultStopTimeout is the wall-clock budget Stop waits for the bus to
// exit after the signal is sent. 5s covers the bus's own graceful tear-down
// (broadcast Bye → consumer goroutines drain → cleanup) with margin.
// DefaultStopTimeout is the wall-clock budget Stop waits for each graceful or
// fallback termination phase. 5s covers the bus's own tear-down (broadcast
// Bye → consumer goroutines drain → cleanup) with margin.
const DefaultStopTimeout = 5 * time.Second
// ErrNotRunning indicates bus.lock either does not exist or its recorded
@@ -33,20 +35,32 @@ const DefaultStopTimeout = 5 * time.Second
// distinguish "nothing to stop" from "failed to stop".
var ErrNotRunning = errors.New("busctl: bus is not running")
// ErrOwnerUnverified indicates the PID from bus.lock is alive but no longer
// owns that lock. Stop refuses to send a process-level signal in this state
// because the operating system may have reused the stale PID.
var ErrOwnerUnverified = errors.New("busctl: bus process ownership could not be verified")
var (
stopReadHolderPID = bus.ReadHolderPID
stopAlive = process.Alive
stopFindProcess = os.FindProcess
stopSignalProcess = func(proc *os.Process, signal os.Signal) error { return proc.Signal(signal) }
stopReadHolderPID = bus.ReadHolderPID
stopValidateHolderOwner = bus.ValidateHolderOwnership
stopAlive = process.Alive
stopFindProcess = os.FindProcess
stopSignalProcess = func(proc *os.Process, signal os.Signal) error { return proc.Signal(signal) }
stopDial = transport.Dial
stopRequest = requestBusStop
stopWaitForBusExit = waitForBusExit
)
// StopConfig identifies the target bus and tunes timing.
type StopConfig struct {
// WorkDir holds bus.lock; Stop reads the PID from there.
WorkDir string
// Timeout is the total wall-clock budget for graceful exit. After this,
// Stop returns an error; it does NOT escalate to SIGKILL — leave that
// to the operator.
// IPCEndpoint enables the cross-platform graceful stop RPC. Callers that
// know the bus identity should always provide it. Empty preserves the
// legacy signal-only fallback for compatibility and focused tests.
IPCEndpoint string
// Timeout is the wall-clock budget for graceful exit and, if required,
// the subsequent platform termination fallback.
Timeout time.Duration
}
@@ -54,13 +68,12 @@ type StopConfig struct {
// for the process to actually die. Returns ErrNotRunning if no bus is
// running for that work dir.
//
// Implementation note: on Unix we send SIGTERM. The bus daemon's Run loop
// watches its parent ctx for cancellation; the cobra `event _bus` command
// wires signal.NotifyContext so SIGTERM triggers ctx.Done() → graceful
// shutdown path. On Windows we use os.Process.Signal(os.Interrupt) which
// the Go runtime maps to TerminateProcess for processes outside our
// console group; for v1 that's acceptable (Windows graceful shutdown is
// future work — plan §16 v2).
// Stop first asks the bus to shut down through its owner-only IPC endpoint.
// This is the normal path on every platform and lets the daemon cancel its
// cloud source, notify consumers, and release its lock. If the endpoint is
// unavailable (for example an older bus) or graceful shutdown times out, the
// platform signal is used as a compatibility fallback: SIGTERM on Unix and
// TerminateProcess via os.Kill on Windows.
func Stop(cfg StopConfig) error {
if cfg.WorkDir == "" {
return errors.New("busctl: StopConfig.WorkDir is required")
@@ -81,6 +94,32 @@ func Stop(cfg StopConfig) error {
if err != nil {
return fmt.Errorf("busctl: find process %d: %w", pid, err)
}
if strings.TrimSpace(cfg.IPCEndpoint) != "" {
if err := stopRequest(cfg.IPCEndpoint); err == nil && stopWaitForBusExit(pid, cfg.Timeout) {
return nil
}
}
// The bus may finish shutting down immediately after the graceful wait
// reaches its deadline (or after a failed IPC request). Recheck before
// inspecting ownership so a completed stop is not reported as stale.
if !stopAlive(pid) {
return nil
}
owner, err := stopValidateHolderOwner(LockPath(cfg.WorkDir), pid)
if err != nil {
return fmt.Errorf("busctl: verify bus pid=%d ownership: %w", pid, err)
}
if !owner {
// ValidateHolderOwnership observes a released lock both for a stale,
// reused PID and for a bus that exits during the ownership probe. Only
// the still-alive case is unsafe to signal.
if !stopAlive(pid) {
return nil
}
return fmt.Errorf("%w: pid=%d does not own %s; stale PID was not signalled", ErrOwnerUnverified, pid, LockPath(cfg.WorkDir))
}
if err := stopSignalProcess(proc, stopSignal()); err != nil {
// On many Unix platforms Signal returns "process already finished"
// when the bus has just exited on its own — treat that as success.
@@ -90,13 +129,45 @@ func Stop(cfg StopConfig) error {
return fmt.Errorf("busctl: signal bus pid=%d: %w", pid, err)
}
// Poll for actual exit.
deadline := time.Now().Add(cfg.Timeout)
for time.Now().Before(deadline) {
if !stopAlive(pid) {
return nil
}
time.Sleep(50 * time.Millisecond)
if stopWaitForBusExit(pid, cfg.Timeout) {
return nil
}
return fmt.Errorf("busctl: bus pid=%d did not exit within %s", pid, cfg.Timeout)
}
func requestBusStop(endpoint string) error {
conn, err := stopDial(endpoint)
if err != nil {
return fmt.Errorf("busctl: dial bus for stop: %w", err)
}
defer conn.Close()
_ = conn.SetDeadline(time.Now().Add(DefaultStatusRPCTimeout))
w := transport.NewWriter(conn)
r := transport.NewReader(conn)
if err := w.WriteJSON(transport.Hello{
Type: transport.FrameTypeHello,
ConsumerPID: os.Getpid(),
Role: transport.HelloRoleStop,
}); err != nil {
return fmt.Errorf("busctl: write stop hello: %w", err)
}
var bye transport.Bye
if err := r.ReadJSON(&bye); err != nil {
return fmt.Errorf("busctl: read stop response: %w", err)
}
if bye.Type != transport.FrameTypeBye || bye.Reason != "stop_request" {
return fmt.Errorf("busctl: unexpected stop response type=%q reason=%q", bye.Type, bye.Reason)
}
return nil
}
func waitForBusExit(pid int, timeout time.Duration) bool {
deadline := time.Now().Add(timeout)
for time.Now().Before(deadline) {
if !stopAlive(pid) {
return true
}
time.Sleep(50 * time.Millisecond)
}
return !stopAlive(pid)
}
+270 -1
View File
@@ -16,6 +16,7 @@ package busctl
import (
"context"
"errors"
"net"
"os"
"os/exec"
"path/filepath"
@@ -23,7 +24,10 @@ import (
"testing"
"time"
dwsevent "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/bus"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/event/transport"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
)
func TestStop_NotRunningWhenLockMissing(t *testing.T) {
@@ -34,7 +38,7 @@ func TestStop_NotRunningWhenLockMissing(t *testing.T) {
}
}
func TestStop_NotRunningWhenPIDDead(t *testing.T) {
func TestCrossPlatformCoverageStopNotRunningWhenPIDDead(t *testing.T) {
dir := shortTempDir(t)
// Write a definitely-dead PID into bus.lock.
if err := os.WriteFile(LockPath(dir), []byte("2147483646\n"), 0o600); err != nil {
@@ -46,6 +50,269 @@ func TestStop_NotRunningWhenPIDDead(t *testing.T) {
}
}
func TestCrossPlatformCoverageStopPrefersGracefulIPC(t *testing.T) {
const pid = 4242
stopped := false
signals := 0
testseam.Swap(t, &stopReadHolderPID, func(string) int { return pid })
testseam.Swap(t, &stopAlive, func(int) bool { return !stopped })
proc, err := os.FindProcess(os.Getpid())
if err != nil {
t.Fatal(err)
}
testseam.Swap(t, &stopFindProcess, func(int) (*os.Process, error) { return proc, nil })
testseam.Swap(t, &stopRequest, func(endpoint string) error {
if endpoint != "test-endpoint" {
t.Fatalf("stop endpoint = %q", endpoint)
}
stopped = true
return nil
})
testseam.Swap(t, &stopSignalProcess, func(*os.Process, os.Signal) error {
signals++
return nil
})
if err := Stop(StopConfig{
WorkDir: "test-workdir",
IPCEndpoint: "test-endpoint",
Timeout: time.Second,
}); err != nil {
t.Fatalf("Stop() = %v", err)
}
if signals != 0 {
t.Fatalf("graceful IPC stop used %d process signals", signals)
}
}
func TestCrossPlatformCoverageStopDoesNotSignalUnverifiedReusedPID(t *testing.T) {
const pid = 4242
testseam.Swap(t, &stopReadHolderPID, func(string) int { return pid })
testseam.Swap(t, &stopAlive, func(int) bool { return true })
proc, err := os.FindProcess(os.Getpid())
if err != nil {
t.Fatal(err)
}
testseam.Swap(t, &stopFindProcess, func(int) (*os.Process, error) { return proc, nil })
testseam.Swap(t, &stopRequest, func(string) error { return errors.New("stale endpoint") })
testseam.Swap(t, &stopValidateHolderOwner, func(path string, gotPID int) (bool, error) {
if path != LockPath("test-workdir") || gotPID != pid {
t.Fatalf("ownership check path=%q pid=%d", path, gotPID)
}
return false, nil
})
signals := 0
testseam.Swap(t, &stopSignalProcess, func(*os.Process, os.Signal) error {
signals++
return nil
})
err = Stop(StopConfig{
WorkDir: "test-workdir",
IPCEndpoint: "stale-endpoint",
Timeout: time.Millisecond,
})
if !errors.Is(err, ErrOwnerUnverified) {
t.Fatalf("Stop() error = %v, want ErrOwnerUnverified", err)
}
if signals != 0 {
t.Fatalf("unverified reused PID received %d signals", signals)
}
}
func TestCrossPlatformCoverageStopAcceptsExitAtGracefulTimeoutBoundary(t *testing.T) {
const pid = 4242
t.Run("before ownership validation", func(t *testing.T) {
exited := false
validated := false
testseam.Swap(t, &stopReadHolderPID, func(string) int { return pid })
testseam.Swap(t, &stopAlive, func(int) bool { return !exited })
proc, err := os.FindProcess(os.Getpid())
if err != nil {
t.Fatal(err)
}
testseam.Swap(t, &stopFindProcess, func(int) (*os.Process, error) { return proc, nil })
testseam.Swap(t, &stopRequest, func(string) error { return nil })
testseam.Swap(t, &stopWaitForBusExit, func(int, time.Duration) bool {
exited = true
return false
})
testseam.Swap(t, &stopValidateHolderOwner, func(string, int) (bool, error) {
validated = true
return false, nil
})
if err := Stop(StopConfig{WorkDir: "test-workdir", IPCEndpoint: "test-endpoint"}); err != nil {
t.Fatalf("Stop() = %v, want success after bus exit", err)
}
if validated {
t.Fatal("ownership was validated after the bus had already exited")
}
})
t.Run("during ownership validation", func(t *testing.T) {
exited := false
signals := 0
testseam.Swap(t, &stopReadHolderPID, func(string) int { return pid })
testseam.Swap(t, &stopAlive, func(int) bool { return !exited })
proc, err := os.FindProcess(os.Getpid())
if err != nil {
t.Fatal(err)
}
testseam.Swap(t, &stopFindProcess, func(int) (*os.Process, error) { return proc, nil })
testseam.Swap(t, &stopRequest, func(string) error { return nil })
testseam.Swap(t, &stopWaitForBusExit, func(int, time.Duration) bool { return false })
testseam.Swap(t, &stopValidateHolderOwner, func(string, int) (bool, error) {
exited = true
return false, nil
})
testseam.Swap(t, &stopSignalProcess, func(*os.Process, os.Signal) error {
signals++
return nil
})
if err := Stop(StopConfig{WorkDir: "test-workdir", IPCEndpoint: "test-endpoint"}); err != nil {
t.Fatalf("Stop() = %v, want success after exit during ownership validation", err)
}
if signals != 0 {
t.Fatalf("exited bus received %d fallback signals", signals)
}
})
}
func TestCrossPlatformCoverageStopReportsOwnershipValidationError(t *testing.T) {
const pid = 4242
errInjected := errors.New("ownership validation failed")
testseam.Swap(t, &stopReadHolderPID, func(string) int { return pid })
testseam.Swap(t, &stopAlive, func(int) bool { return true })
proc, err := os.FindProcess(os.Getpid())
if err != nil {
t.Fatal(err)
}
testseam.Swap(t, &stopFindProcess, func(int) (*os.Process, error) { return proc, nil })
testseam.Swap(t, &stopValidateHolderOwner, func(string, int) (bool, error) {
return false, errInjected
})
signals := 0
testseam.Swap(t, &stopSignalProcess, func(*os.Process, os.Signal) error {
signals++
return nil
})
err = Stop(StopConfig{WorkDir: "test-workdir"})
if !errors.Is(err, errInjected) {
t.Fatalf("Stop() error = %v, want injected ownership error", err)
}
if signals != 0 {
t.Fatalf("ownership validation error sent %d signals", signals)
}
}
func TestCrossPlatformCoverageRequestBusStopProtocol(t *testing.T) {
dir := shortTempDir(t)
endpoint := dwsevent.IPCEndpoint(
dir,
"open",
dwsevent.SourceKindAppStream,
dwsevent.IdentityHash(dir),
)
listener, err := transport.Listen(endpoint)
if err != nil {
t.Fatalf("Listen() = %v", err)
}
defer listener.Close()
serverDone := make(chan error, 1)
go func() {
conn, err := listener.Accept()
if err != nil {
serverDone <- err
return
}
defer conn.Close()
r := transport.NewReader(conn)
w := transport.NewWriter(conn)
var hello transport.Hello
if err := r.ReadJSON(&hello); err != nil {
serverDone <- err
return
}
if hello.Type != transport.FrameTypeHello || hello.Role != transport.HelloRoleStop {
serverDone <- errors.New("unexpected stop hello")
return
}
serverDone <- w.WriteJSON(transport.Bye{
Type: transport.FrameTypeBye,
Reason: "stop_request",
})
}()
if err := requestBusStop(endpoint); err != nil {
t.Fatalf("requestBusStop() = %v", err)
}
if err := <-serverDone; err != nil {
t.Fatalf("stop protocol server = %v", err)
}
}
func TestCrossPlatformCoverageRequestBusStopErrors(t *testing.T) {
t.Run("dial", func(t *testing.T) {
testseam.Swap(t, &stopDial, func(string) (net.Conn, error) {
return nil, errors.New("dial failed")
})
if err := requestBusStop("test-endpoint"); err == nil {
t.Fatal("requestBusStop() unexpectedly succeeded")
}
})
t.Run("write", func(t *testing.T) {
testseam.Swap(t, &stopDial, func(string) (net.Conn, error) {
return &queryErrorConn{failAt: 1}, nil
})
if err := requestBusStop("test-endpoint"); err == nil {
t.Fatal("requestBusStop() unexpectedly succeeded")
}
})
t.Run("read", func(t *testing.T) {
testseam.Swap(t, &stopDial, func(string) (net.Conn, error) {
return &queryErrorConn{}, nil
})
if err := requestBusStop("test-endpoint"); err == nil {
t.Fatal("requestBusStop() unexpectedly succeeded")
}
})
t.Run("unexpected response", func(t *testing.T) {
client, server := net.Pipe()
t.Cleanup(func() {
_ = client.Close()
_ = server.Close()
})
testseam.Swap(t, &stopDial, func(string) (net.Conn, error) {
return client, nil
})
serverDone := make(chan error, 1)
go func() {
var hello transport.Hello
if err := transport.NewReader(server).ReadJSON(&hello); err != nil {
serverDone <- err
return
}
serverDone <- transport.NewWriter(server).WriteJSON(transport.Bye{
Type: transport.FrameTypeBye,
Reason: "unexpected",
})
}()
if err := requestBusStop("test-endpoint"); err == nil {
t.Fatal("requestBusStop() unexpectedly succeeded")
}
if err := <-serverDone; err != nil {
t.Fatalf("stop protocol server = %v", err)
}
})
}
func TestStop_SignalsLiveProcess(t *testing.T) {
skipOnWindows(t)
dir := shortTempDir(t)
@@ -63,6 +330,7 @@ func TestStop_SignalsLiveProcess(t *testing.T) {
}
}()
pid := cmd.Process.Pid
testseam.Swap(t, &stopValidateHolderOwner, func(string, int) (bool, error) { return true, nil })
// Reap the child in background so Wait doesn't leave a zombie.
waited := make(chan error, 1)
@@ -111,6 +379,7 @@ func TestStop_TimeoutWhenChildIgnoresSignal(t *testing.T) {
_, _ = cmd.Process.Wait()
}()
pid := cmd.Process.Pid
testseam.Swap(t, &stopValidateHolderOwner, func(string, int) (bool, error) { return true, nil })
if err := os.WriteFile(LockPath(dir), []byte(strconv.Itoa(pid)+"\n"), 0o600); err != nil {
t.Fatal(err)
+4 -5
View File
@@ -17,8 +17,7 @@ package busctl
import "os"
// stopSignal returns the graceful-shutdown signal for Windows. The Go
// runtime maps os.Interrupt to TerminateProcess for non-console-group
// processes — not truly graceful, but acceptable for v1 (Windows graceful
// shutdown via Ctrl+Break is in the v2 backlog, plan §16).
func stopSignal() os.Signal { return os.Interrupt }
// stopSignal is only the fallback after the graceful IPC stop path fails or
// times out. Go maps os.Kill to TerminateProcess on Windows; os.Interrupt is
// unsupported and returns syscall.EWINDOWS.
func stopSignal() os.Signal { return os.Kill }
@@ -0,0 +1,27 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//go:build windows
package busctl
import (
"os"
"testing"
)
func TestCrossPlatformCoverageWindowsStopFallbackUsesKill(t *testing.T) {
if stopSignal() != os.Kill {
t.Fatalf("stopSignal() = %v, want os.Kill", stopSignal())
}
}
@@ -0,0 +1,249 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package main
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"io"
"os"
"path/filepath"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/app"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cli"
"github.com/spf13/cobra"
)
type workflowFixture struct {
Version int `json:"version"`
Description string `json:"description,omitempty"`
Cases []cli.ToolSearchWorkflowEvaluationCase `json:"cases"`
}
type comparisonOutput struct {
Diagnostic cli.ToolSearchDiagnosticComparison `json:"diagnostic"`
Shadows []cli.ToolSearchDiagnosticComparison `json:"shadows"`
AgentPlanningAB *cli.ToolSearchAgentPlanningABReport `json:"agent_planning_ab,omitempty"`
AgentAB *cli.ToolSearchAgentABReport `json:"agent_ab,omitempty"`
Independent *cli.ToolSearchIndependentReport `json:"independent,omitempty"`
}
type independentFixture struct {
Version string `json:"version"`
State string `json:"state"`
Description string `json:"description,omitempty"`
Cases []cli.ToolSearchIndependentCase `json:"cases"`
}
func main() {
var workflowsPath string
var agentResultsPath string
var directPlansPath string
var searchPlansPath string
var agentModel string
var outputPath string
var independentQrelsPath string
flag.StringVar(&workflowsPath, "workflows", "scripts/testdata/tool_search_workflows.json", "reviewed diagnostic workflow fixture")
flag.StringVar(&agentResultsPath, "agent-results", "", "optional paired Agent A/B result JSON")
flag.StringVar(&directPlansPath, "agent-plans-direct", "", "optional direct_schema Agent planning output JSON")
flag.StringVar(&searchPlansPath, "agent-plans-search", "", "optional search_inspect Agent planning output JSON")
flag.StringVar(&agentModel, "agent-model", "", "model identifier for paired planning outputs")
flag.StringVar(&outputPath, "output", "", "output JSON path; stdout when empty")
flag.StringVar(&independentQrelsPath, "independent-qrels", "", "optional independently authored tool-search-qrels.v1 JSON")
flag.Parse()
// Runtime and CI use the same declaration-owned Catalog assembly. The
// comparison generator never reads or writes a repository Catalog JSON.
cli.RegisterSchemaSourceRoot(func() *cobra.Command { return app.NewSchemaSourceRootCommand() })
workflows, err := readWorkflows(workflowsPath)
if err != nil {
fatal(err)
}
diagnostic, err := cli.BuildDeliveryToolSearchDiagnosticComparison(context.Background(), workflows)
if err != nil {
fatal(err)
}
actionShadow, err := cli.BuildDeliveryToolSearchDiagnosticComparisonForAlgorithm(context.Background(), workflows, cli.ToolSearchLexicalBM25Action)
if err != nil {
fatal(err)
}
shadow, err := cli.BuildDeliveryToolSearchDiagnosticComparisonForAlgorithm(context.Background(), workflows, cli.ToolSearchLexicalTFIDF)
if err != nil {
fatal(err)
}
result := comparisonOutput{Diagnostic: diagnostic, Shadows: []cli.ToolSearchDiagnosticComparison{actionShadow, shadow}}
if independentQrelsPath != "" {
fixture, readErr := readIndependentQrels(independentQrelsPath)
if readErr != nil {
fatal(readErr)
}
independent, evaluationErr := cli.BuildDeliveryToolSearchIndependentEvaluation(context.Background(), fixture.Cases)
if evaluationErr != nil {
fatal(evaluationErr)
}
result.Independent = &independent
}
if directPlansPath != "" || searchPlansPath != "" || agentModel != "" {
if directPlansPath == "" || searchPlansPath == "" || agentModel == "" {
fatal(fmt.Errorf("-agent-plans-direct, -agent-plans-search and -agent-model must be provided together"))
}
directPlans, readErr := readAgentPlans(directPlansPath)
if readErr != nil {
fatal(readErr)
}
searchPlans, readErr := readAgentPlans(searchPlansPath)
if readErr != nil {
fatal(readErr)
}
planningReport, scoreErr := cli.ScoreToolSearchAgentPlanningAB(agentModel, workflows, directPlans, searchPlans)
if scoreErr != nil {
fatal(scoreErr)
}
result.AgentPlanningAB = &planningReport
}
if agentResultsPath != "" {
agentInput, readErr := readAgentResults(agentResultsPath)
if readErr != nil {
fatal(readErr)
}
if agentInput.Catalog != diagnostic.Catalog {
fatal(fmt.Errorf("Agent A/B Catalog hashes do not match the embedded binary"))
}
agentReport, aggregateErr := cli.AggregateToolSearchAgentAB(agentInput)
if aggregateErr != nil {
fatal(aggregateErr)
}
diagnostic.AgentABStatus = "completed"
result.Diagnostic = diagnostic
result.AgentAB = &agentReport
}
encoded, err := json.MarshalIndent(result, "", " ")
if err != nil {
fatal(err)
}
encoded = append(encoded, '\n')
if outputPath == "" {
if _, err := os.Stdout.Write(encoded); err != nil {
fatal(err)
}
return
}
if err := writeAtomically(outputPath, encoded); err != nil {
fatal(err)
}
}
func readIndependentQrels(path string) (independentFixture, error) {
data, err := os.ReadFile(path)
if err != nil {
return independentFixture{}, fmt.Errorf("read independent qrels: %w", err)
}
var fixture independentFixture
if err := decodeStrict(data, &fixture); err != nil {
return independentFixture{}, fmt.Errorf("decode independent qrels: %w", err)
}
if fixture.Version != "tool-search-qrels.v1" || len(fixture.Cases) == 0 {
return independentFixture{}, fmt.Errorf("independent qrels must be tool-search-qrels.v1 and non-empty")
}
return fixture, nil
}
func readWorkflows(path string) ([]cli.ToolSearchWorkflowEvaluationCase, error) {
data, err := os.ReadFile(path)
if err != nil {
return nil, fmt.Errorf("read workflows: %w", err)
}
var fixture workflowFixture
if err := decodeStrict(data, &fixture); err != nil {
return nil, fmt.Errorf("decode workflows: %w", err)
}
if fixture.Version != 1 || len(fixture.Cases) == 0 {
return nil, fmt.Errorf("workflow fixture must be version 1 and non-empty")
}
return fixture.Cases, nil
}
func readAgentResults(path string) (cli.ToolSearchAgentABInput, error) {
data, err := os.ReadFile(path)
if err != nil {
return cli.ToolSearchAgentABInput{}, fmt.Errorf("read Agent A/B results: %w", err)
}
var input cli.ToolSearchAgentABInput
if err := decodeStrict(data, &input); err != nil {
return cli.ToolSearchAgentABInput{}, fmt.Errorf("decode Agent A/B results: %w", err)
}
return input, nil
}
func readAgentPlans(path string) (cli.ToolSearchAgentPlanOutput, error) {
data, err := os.ReadFile(path)
if err != nil {
return cli.ToolSearchAgentPlanOutput{}, fmt.Errorf("read Agent plans: %w", err)
}
var output cli.ToolSearchAgentPlanOutput
if err := decodeStrict(data, &output); err != nil {
return cli.ToolSearchAgentPlanOutput{}, fmt.Errorf("decode Agent plans: %w", err)
}
return output, nil
}
func decodeStrict(data []byte, target any) error {
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.DisallowUnknownFields()
if err := decoder.Decode(target); err != nil {
return err
}
var extra any
if err := decoder.Decode(&extra); err != io.EOF {
if err == nil {
return fmt.Errorf("multiple JSON values are not allowed")
}
return err
}
return nil
}
func writeAtomically(path string, data []byte) error {
directory := filepath.Dir(path)
if err := os.MkdirAll(directory, 0o755); err != nil {
return err
}
temporary, err := os.CreateTemp(directory, ".tool-search-comparison-*.tmp")
if err != nil {
return err
}
temporaryPath := temporary.Name()
defer os.Remove(temporaryPath)
if _, err := temporary.Write(data); err != nil {
temporary.Close()
return err
}
if err := temporary.Sync(); err != nil {
temporary.Close()
return err
}
if err := temporary.Close(); err != nil {
return err
}
return os.Rename(temporaryPath, path)
}
func fatal(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
+82 -2
View File
@@ -230,7 +230,7 @@ func newCalendarCommand() *cobra.Command {
Long: `管理钉钉日历:日程、参会人、会议室、闲忙、附件、日历本、访问权限。调用前必须先使用 --help 查看参数结构。
命令结构:
dws calendar event [list|get|create|update|delete|suggest|respond|instances] 日程管理
dws calendar event [list|get|create|update|delete|suggest|respond|instances|share-info] 日程管理
dws calendar attendee [list|add|delete] 参会人管理
dws calendar room [search|add|delete|list-groups] 会议室管理
dws calendar busy search 闲忙查询 (可查人、查会议室)
@@ -2259,7 +2259,87 @@ func newCalendarCommand() *cobra.Command {
eventInstancesCmd.Flags().Int("count", 0, "")
_ = eventInstancesCmd.Flags().MarkHidden("count")
eventCmd.AddCommand(eventListCmd, eventGetCmd, eventCreateCmd, eventUpdateCmd, eventDeleteCmd, eventSuggestCmd, eventRespondCmd, eventInstancesCmd)
eventShareInfoCmd := &cobra.Command{
Use: "share-info",
Aliases: []string{"share_info"},
Short: "获取日程的分享信息",
Long: `根据日程 ID 获取日程的分享信息,展示日程主题、组织人、地点、入会信息等,用于向他人分享日程。eventId 可通过 dws calendar event list 获取。`,
Example: ` dws calendar event share-info --id EVENT_ID
dws calendar event share-info --id EVENT_ID --language zh-CN
dws calendar event share-info --id EVENT_ID --calendar-id primary`,
RunE: func(cmd *cobra.Command, args []string) error {
eventID, err := mustFlagOrFallback(cmd, "id", "event", "event-id", "eventId")
if err != nil {
return err
}
toolArgs := map[string]any{"eventId": eventID}
if v := flagOrFallback(cmd, "calendar-id", "calendarId", "calendar"); v != "" {
toolArgs["calendarId"] = v
}
if v := flagOrFallback(cmd, "language", "lang"); v != "" {
toolArgs["language"] = v
}
return callMCPTool("get_event_share_info", toolArgs)
},
}
DeclareLeafMetadata(eventShareInfoCmd, LeafSpec{
Safety: contract.SafetySpec{
Effect: "read", Risk: "low",
Confirmation: "not_required", Idempotency: "idempotent",
},
Contract: LeafContract{
Identity: contract.ToolIdentitySpec{
ProductID: "calendar",
Name: "get_event_share_info",
CanonicalPath: "calendar.get_event_share_info",
CLIPath: "calendar event share-info",
PrimaryCLIPath: "calendar event share-info",
Aliases: []string{"calendar event share_info"},
},
Description: "获取日程的分享信息",
Interface: &contract.InterfaceSpec{
Mode: "mcp",
Availability: "available",
Ref: &contract.InterfaceRefSpec{ProductID: "calendar", RPCName: "get_event_share_info"},
},
Selection: contract.SelectionSpec{
AgentSummary: "获取日程的分享信息(主题、组织人、地点、入会信息等)",
UseWhen: []string{"已知 eventId,需要获取日程分享信息或入会信息用于分享给他人时"},
AvoidWhen: []string{
"要查看日程详情改用 dws calendar event get",
"未知 eventId 时先 dws calendar event list",
},
Examples: []string{
"dws calendar event share-info --id <EVENT_ID>",
"dws calendar event share-info --id <EVENT_ID> --language zh-CN",
},
},
Parameters: []contract.ParamDecl{
{Name: "id", Property: "eventId", Required: boolPtr(true)},
{Name: "calendar-id", Property: "calendarId"},
{Name: "language", Property: "language"},
},
},
})
// EventShareInfo flags
eventShareInfoCmd.Flags().String("id", "", "日程 ID (必填)")
eventShareInfoCmd.Flags().String("event", "", "")
_ = eventShareInfoCmd.Flags().MarkHidden("event")
eventShareInfoCmd.Flags().String("event-id", "", "")
_ = eventShareInfoCmd.Flags().MarkHidden("event-id")
eventShareInfoCmd.Flags().String("eventId", "", "")
_ = eventShareInfoCmd.Flags().MarkHidden("eventId")
eventShareInfoCmd.Flags().String("calendar-id", "", "日历 ID (默认 primary 主日历)")
eventShareInfoCmd.Flags().String("calendarId", "", "")
_ = eventShareInfoCmd.Flags().MarkHidden("calendarId")
eventShareInfoCmd.Flags().String("calendar", "", "")
_ = eventShareInfoCmd.Flags().MarkHidden("calendar")
eventShareInfoCmd.Flags().String("language", "", "语言代码 (可选,如 zh-CN)")
eventShareInfoCmd.Flags().String("lang", "", "")
_ = eventShareInfoCmd.Flags().MarkHidden("lang")
eventCmd.AddCommand(eventListCmd, eventGetCmd, eventCreateCmd, eventUpdateCmd, eventDeleteCmd, eventSuggestCmd, eventRespondCmd, eventInstancesCmd, eventShareInfoCmd)
// participant
participantCmd.PersistentFlags().String("event", "", "日程 ID (必填)")
+54 -8
View File
@@ -3413,10 +3413,11 @@ func newChatCommand() *cobra.Command {
chatMessageSendByBotCmd := &cobra.Command{
Use: "send-by-bot",
Short: "机器人发送消息(--conversation-id 群聊 / --users 单聊)",
Long: `群聊:传 --conversation-id 指定群;单聊:传 --users 或 --open-dingtalk-ids 指定用户列表,与 --conversation-id 只能选其一,不能同时指定。省略 --msg-type 时发送 Markdown;图片使用 --msg-type image --image-url;本地文件使用 --msg-type file --file-path,CLI 完成上传后发送。
Long: `群聊:传 --conversation-id 指定群;单聊:传 --users 或 --open-dingtalk-ids 指定用户列表,与 --conversation-id 只能选其一,不能同时指定。省略 --msg-type 时发送 Markdown;图片使用 --msg-type image --image-url;本地文件使用 --msg-type file --file-path,CLI 完成上传后发送。机器人引用回复仅支持群聊 Markdown,同时传 --reply(原消息 openMessageId)和 --ref-sender(原消息发送者 openDingTalkId);回复可省略 --title,CLI 会从正文生成标题。
⚠️ 重要:该接口会真实发送消息到目标会话,不可用于测试或试探性调用。调用前必须确认消息内容和接收对象无误。`,
Example: ` dws chat message send-by-bot --robot-code <robot-code> --conversation-id <openconversation_id> --title "日报" --text "## 今日完成..."
dws chat message send-by-bot --robot-code <robot-code> --conversation-id <openconversation_id> --reply <openMessageId> --ref-sender <openDingTalkId> --text "收到"
dws chat message send-by-bot --robot-code <robot-code> --conversation-id <openconversation_id> --msg-type image --image-url "https://example.com/image.png"
dws chat message send-by-bot --robot-code <robot-code> --conversation-id <openconversation_id> --msg-type file --file-path ./report.pdf
dws chat message send-by-bot --robot-code <robot-code> --users userId1,userId2 --title "提醒" --text "请提交周报"
@@ -3433,10 +3434,17 @@ func newChatCommand() *cobra.Command {
text := mustGetFlag(cmd, "text")
imageURL := strings.TrimSpace(mustGetFlag(cmd, "image-url"))
filePath := mustGetFlag(cmd, "file-path")
referenceOpenMessageID := strings.TrimSpace(mustGetFlag(cmd, "reply"))
refSender := strings.TrimSpace(mustGetFlag(cmd, "ref-sender"))
replyChanged := cmd.Flags().Changed("reply")
refSenderChanged := cmd.Flags().Changed("ref-sender")
if err := validateRequiredFlags(cmd, "robot-code"); err != nil {
return err
}
if replyChanged != refSenderChanged || (replyChanged && (referenceOpenMessageID == "" || refSender == "")) {
return fmt.Errorf("--reply and --ref-sender must be specified together")
}
if msgType == "" {
if imageURL != "" {
return fmt.Errorf("--msg-type image is required when using --image-url")
@@ -3448,8 +3456,17 @@ func newChatCommand() *cobra.Command {
}
switch msgType {
case "markdown":
if err := validateRequiredFlags(cmd, "title", "text"); err != nil {
return err
if referenceOpenMessageID != "" {
if err := validateRequiredFlags(cmd, "text"); err != nil {
return err
}
if title == "" {
title = sanitizeTitleFromText(text)
}
} else {
if err := validateRequiredFlags(cmd, "title", "text"); err != nil {
return err
}
}
case "image":
if imageURL == "" {
@@ -3475,6 +3492,22 @@ func newChatCommand() *cobra.Command {
return fmt.Errorf("--conversation-id or --users/--open-dingtalk-ids is required")
}
if referenceOpenMessageID != "" {
if chatID == "" {
return fmt.Errorf("--reply and --ref-sender are only supported with --conversation-id")
}
if msgType != "markdown" {
return fmt.Errorf("--reply and --ref-sender only support Markdown group messages")
}
if !isOpenDingTalkID(refSender) {
resolved, err := resolveOpenDingTalkID(cmd.Context(), refSender)
if err != nil {
return err
}
refSender = resolved
}
}
userIDs := splitCommaList(usersStr)
openDingTalkIDs := splitCommaList(openDingtalkIdsStr)
fileURL := ""
@@ -3540,6 +3573,10 @@ func newChatCommand() *cobra.Command {
toolArgs["msgKey"] = "sampleMarkdownDX"
}
toolArgs["openConversationId"] = chatID
if referenceOpenMessageID != "" {
toolArgs["referenceOpenMessageId"] = referenceOpenMessageID
toolArgs["srcMsgSendOpenDingTalkId"] = refSender
}
if len(atUserIds) > 0 {
toolArgs["atUserIds"] = atUserIds
}
@@ -3586,17 +3623,20 @@ func newChatCommand() *cobra.Command {
CLIPath: "chat message send-by-bot",
PrimaryCLIPath: "chat message send-by-bot",
},
Description: "以应用机器人身份发送 Markdown、图片或文件群消息或批量单聊",
Description: "以应用机器人身份发送 Markdown、图片或文件群消息、群聊引用回复或批量单聊",
Interface: &contract.InterfaceSpec{
Mode: "composite",
Availability: "available",
Reason: "命令包含多个 RPC、条件分派或本地 HTTP/文件步骤,不能绑定为单一 interface_ref",
},
Selection: contract.SelectionSpec{
AgentSummary: "以应用机器人身份发送 Markdown、图片或文件群消息或批量单聊",
UseWhen: []string{"已有 robotCode 且需要机器人身份投递 Markdown、图片或文件时"},
AgentSummary: "以应用机器人身份发送 Markdown、图片或文件群消息、群聊引用回复或批量单聊",
UseWhen: []string{"已有 robotCode 且需要机器人身份投递 Markdown、图片或文件时", "需要机器人在群聊中引用回复一条已有消息时"},
AvoidWhen: []string{"个人身份发送或自定义 Webhook 告警不要使用"},
Examples: []string{"dws chat message send-by-bot --robot-code <robotCode> --group <openConversationId> --title \"日报\" --text \"今日进展\""},
Examples: []string{
"dws chat message send-by-bot --robot-code <robotCode> --group <openConversationId> --title \"日报\" --text \"今日进展\"",
"dws chat message send-by-bot --robot-code <robotCode> --group <openConversationId> --reply <openMessageId> --ref-sender <openDingTalkId> --text \"收到\"",
},
},
// Keep title/text required_when out of Schema: the runtime switch above
// enforces Markdown inputs, while adding it breaks merge-base compatibility.
@@ -3605,6 +3645,8 @@ func newChatCommand() *cobra.Command {
{Name: "msg-type", RequiredWhen: "image-url or file-path is provided", Enum: []string{"markdown", "image", "file"}},
{Name: "image-url", RequiredWhen: "msg-type is image"},
{Name: "file-path", RequiredWhen: "msg-type is file"},
{Name: "reply", Property: "referenceOpenMessageId", RequiredWhen: "ref-sender is provided", InterfaceType: "string"},
{Name: "ref-sender", Property: "srcMsgSendOpenDingTalkId", RequiredWhen: "reply is provided", InterfaceType: "string"},
},
},
})
@@ -4768,7 +4810,7 @@ chat message edit 或 chat message recall 的 --message-id 和 --conversation-id
chatMessageSendByBotCmd.Flags().String("conversation-id", "", "群聊 openConversationId(群聊时必填)")
chatMessageSendByBotCmd.Flags().String("users", "", "用户 userId 列表,逗号分隔,最多20个(单聊时必填)")
chatMessageSendByBotCmd.Flags().String("msg-type", "", "消息类型: markdown/image/file(省略时为 markdown;图片使用 image --image-url;本地文件使用 file --file-path)")
chatMessageSendByBotCmd.Flags().String("title", "", "Markdown 消息标题(发送 Markdown 时必填)")
chatMessageSendByBotCmd.Flags().String("title", "", "Markdown 消息标题(发送普通 Markdown 时必填;引用回复省略时从正文生成)")
chatMessageSendByBotCmd.Flags().String("text", "", "Markdown 消息内容(发送 Markdown 时必填;稳定换行用空行,转义形式写 \\n\\n,不要只写 \\n)")
chatMessageSendByBotCmd.Flags().String("image-url", "", "公网图片 URL(msgType=image 时必填)")
chatMessageSendByBotCmd.Flags().String("file-path", "", "本地文件路径(msgType=file 时直接上传并按 file 消息发送)")
@@ -4776,9 +4818,13 @@ chat message edit 或 chat message recall 的 --message-id 和 --conversation-id
chatMessageSendByBotCmd.Flags().String("open-dingtalk-ids", "", "用户 openDingtalkId 列表,逗号分隔(单聊时可替代 --users,可选)")
chatMessageSendByBotCmd.Flags().String("at-open-dingtalk-ids", "", "@指定成员的 openDingtalkId 列表,逗号分隔(仅群聊时生效,可选)")
chatMessageSendByBotCmd.Flags().Bool("at-all", false, "@所有人(可选),服务端接收字符串 true/false")
chatMessageSendByBotCmd.Flags().String("reply", "", "被引用消息的 openMessageId(仅群聊 Markdown;必须与 --ref-sender 同时使用)")
chatMessageSendByBotCmd.Flags().String("ref-sender", "", "被引用消息发送者的 openDingTalkId(仅群聊 Markdown;必须与 --reply 同时使用)")
chatMessageSendByBotCmd.MarkFlagsRequiredTogether("reply", "ref-sender")
cli.AnnotateRuntimeConstraints(chatMessageSendByBotCmd, cli.RuntimeSchemaConstraints{
MutuallyExclusive: [][]string{{"group", "users"}},
RequireOneOf: [][]string{{"group", "users"}},
RequireTogether: [][]string{{"reply", "ref-sender"}},
})
cli.AnnotateRuntimeFlagFormat(chatMessageSendByBotCmd, "file-path", "file-path")
@@ -446,6 +446,100 @@ func TestCrossPlatformCoverageChatBotRichMediaCoverage(t *testing.T) {
if caller.tool != "batch_send_robot_msg_to_users" || caller.args["msgType"] != "sampleMarkdownDX" {
t.Fatalf("direct markdown call = %s %#v", caller.tool, caller.args)
}
if _, exists := caller.args["referenceOpenMessageId"]; exists {
t.Fatalf("ordinary direct message unexpectedly contains reply fields: %#v", caller.args)
}
})
t.Run("group Markdown reference reply", func(t *testing.T) {
const senderOpenDingTalkID = "DAAAAAAAAAAAiE"
caller := &scriptedToolCaller{}
if err := runChatCoverageCommand(t, caller,
"message", "send-by-bot", "--robot-code", "robot", "--conversation-id", "group",
"--reply", "message-id", "--ref-sender", senderOpenDingTalkID,
"--text", "received",
); err != nil {
t.Fatal(err)
}
if caller.tool != "send_robot_group_message" ||
caller.args["referenceOpenMessageId"] != "message-id" ||
caller.args["srcMsgSendOpenDingTalkId"] != senderOpenDingTalkID ||
caller.args["title"] != "received" {
t.Fatalf("group reference reply call = %s %#v", caller.tool, caller.args)
}
})
t.Run("group reference reply resolves sender user ID", func(t *testing.T) {
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{text: `{"result":[{"userId":"sender-user","openDingTalkId":"D-sender"}]}`},
{text: `{}`},
}}
if err := runChatCoverageCommand(t, caller,
"message", "send-by-bot", "--robot-code", "robot", "--conversation-id", "group",
"--reply", "message-id", "--ref-sender", "sender-user",
"--title", "reply", "--text", "received",
); err != nil {
t.Fatal(err)
}
if caller.tool != "send_robot_group_message" || caller.args["srcMsgSendOpenDingTalkId"] != "D-sender" {
t.Fatalf("resolved group reference reply = %s %#v", caller.tool, caller.args)
}
})
t.Run("direct RunE enforces paired reply flags", func(t *testing.T) {
err := runChatCoverageDirect(t, []string{"message", "send-by-bot"}, map[string]string{
"robot-code": "robot",
"conversation-id": "group",
"reply": "message-id",
"title": "reply",
"text": "received",
})
if err == nil || !strings.Contains(err.Error(), "must be specified together") {
t.Fatalf("direct RunE paired reply flags error = %v", err)
}
})
for _, tc := range []struct {
name string
args []string
want string
}{
{name: "reply requires sender", args: []string{"--reply", "message-id"}, want: "ref-sender"},
{name: "sender requires reply", args: []string{"--ref-sender", "D-sender"}, want: "reply"},
{name: "paired reply flags reject empty values", args: []string{"--reply=", "--ref-sender="}, want: "specified together"},
} {
t.Run(tc.name, func(t *testing.T) {
args := []string{
"message", "send-by-bot", "--robot-code", "robot", "--conversation-id", "group",
"--title", "reply", "--text", "received",
}
err := runChatCoverageCommand(t, &scriptedToolCaller{}, append(args, tc.args...)...)
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("paired reply flags error = %v, want %q", err, tc.want)
}
})
}
t.Run("reference reply is group only", func(t *testing.T) {
err := runChatCoverageCommand(t, &scriptedToolCaller{},
"message", "send-by-bot", "--robot-code", "robot", "--users", "user",
"--reply", "message-id", "--ref-sender", "D-sender",
"--title", "reply", "--text", "received",
)
if err == nil || !strings.Contains(err.Error(), "only supported with --conversation-id") {
t.Fatalf("direct reference reply error = %v", err)
}
})
t.Run("reference reply rejects rich media", func(t *testing.T) {
err := runChatCoverageCommand(t, &scriptedToolCaller{},
"message", "send-by-bot", "--robot-code", "robot", "--conversation-id", "group",
"--reply", "message-id", "--ref-sender", "D-sender",
"--msg-type", "image", "--image-url", "https://example.test/image.png",
)
if err == nil || !strings.Contains(err.Error(), "only support Markdown") {
t.Fatalf("rich-media reference reply error = %v", err)
}
})
t.Run("direct image", func(t *testing.T) {
+281 -6
View File
@@ -9,6 +9,7 @@ import (
"net/url"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
@@ -492,10 +493,11 @@ func defaultHTTPGetFile(ctx context.Context, url string, headers map[string]stri
return nil
}
// runMediaInsert implements the three-step flow for inserting an attachment into a document:
// runMediaInsert implements the four-step flow for inserting an attachment into a document:
// 1. get_doc_attachment_upload_info → obtain uploadUrl + resourceId
// 2. HTTP PUT file content to OSS
// 3. insert_document_block with attachment element
// 4. list_document_blocks → prove the uploaded resource is visible in the document
func runMediaInsert(cmd *cobra.Command, _ []string) error {
nodeID, err := mustFlagOrFallback(cmd, "node", "url", "id", "node-id", "doc-id", "file-id")
if err != nil {
@@ -551,7 +553,7 @@ func runMediaInsert(cmd *cobra.Command, _ []string) error {
ctx := cmd.Context()
// Step 1: get upload credentials (uploadUrl + resourceId)
deps.Out.PrintInfo(fmt.Sprintf("[1/3] 获取附件上传凭证 (%s, %d bytes)...", fileName, fileSize))
deps.Out.PrintInfo(fmt.Sprintf("[1/4] 获取附件上传凭证 (%s, %d bytes)...", fileName, fileSize))
credText, err := callMCPToolReturnText(ctx, "get_doc_attachment_upload_info", map[string]any{
"nodeId": nodeID,
@@ -569,7 +571,7 @@ func runMediaInsert(cmd *cobra.Command, _ []string) error {
}
// Step 2: HTTP PUT file to OSS
deps.Out.PrintInfo("[2/3] 上传文件到 OSS...")
deps.Out.PrintInfo("[2/4] 上传文件到 OSS...")
ossHeaders := map[string]string{
"Content-Type": mimeType,
@@ -592,7 +594,7 @@ func runMediaInsert(cmd *cobra.Command, _ []string) error {
}
// Step 3: insert block into document
deps.Out.PrintInfo("[3/3] 插入块到文档...")
deps.Out.PrintInfo("[3/4] 插入块到文档...")
const maxInlineImageSize = 20 * 1024 * 1024 // 20MB
@@ -644,7 +646,8 @@ func runMediaInsert(cmd *cobra.Command, _ []string) error {
insertArgs["referenceBlockId"] = v
}
if err := callMCPTool("insert_document_block", insertArgs); err != nil {
insertText, err := callMCPToolReturnText(ctx, "insert_document_block", insertArgs)
if err != nil {
return apperrors.NewAPI(
"附件已上传,但正文 block 插入结果未知;请先检查媒体列表,不要重复上传或插入",
apperrors.WithOperation("doc.media_insert"),
@@ -665,6 +668,23 @@ func runMediaInsert(cmd *cobra.Command, _ []string) error {
apperrors.WithCause(err),
)
}
insertResult := map[string]any{}
if strings.TrimSpace(insertText) != "" {
if err := json.Unmarshal([]byte(insertText), &insertResult); err != nil {
return docMediaInsertVerificationError(nodeID, resourceID, resourceURL, fileName,
fmt.Errorf("解析 insert_document_block 响应失败: %w", err))
}
}
insertedBlockID := insertedDocBlockID(insertResult)
deps.Out.PrintInfo("[4/4] 回读验证媒体块...")
verifiedBlockID, verifyErr := verifyInsertedDocMedia(ctx, nodeID, insertedBlockID, resourceID, resourceURL)
if verifyErr != nil {
return docMediaInsertVerificationError(nodeID, resourceID, resourceURL, fileName, verifyErr)
}
if insertedBlockID == "" {
insertedBlockID = verifiedBlockID
}
return deps.Out.PrintJSON(map[string]any{
"contractVersion": "doc.operation.v1",
@@ -674,16 +694,271 @@ func runMediaInsert(cmd *cobra.Command, _ []string) error {
"operation": "doc.media_insert",
"data": map[string]any{
"nodeId": nodeID, "resourceId": resourceID, "resourceUrl": resourceURL,
"fileName": fileName, "mimeType": mimeType, "sizeBytes": fileSize, "inserted": true,
"blockId": insertedBlockID, "fileName": fileName, "mimeType": mimeType, "sizeBytes": fileSize,
"inserted": true, "verified": true,
},
"steps": []map[string]any{
{"name": "resolve_upload", "status": "success"},
{"name": "upload_oss", "status": "success"},
{"name": "insert_block", "status": "success"},
{"name": "verify", "status": "success"},
},
})
}
func docMediaInsertVerificationError(nodeID, resourceID, resourceURL, fileName string, cause error) error {
return apperrors.NewAPI(
"附件已上传且插块请求已执行,但回读未能证明媒体块落库;不要直接重试上传或插入",
apperrors.WithOperation("doc.media_insert"),
apperrors.WithReason("doc_media_insert_verification_failed"),
apperrors.WithFailureStage("verify"),
apperrors.WithExecutionStarted(true),
apperrors.WithRetryable(false),
apperrors.WithActions("运行 dws doc +media-list 检查 resourceId", "确认媒体不存在后再决定是否重新执行"),
apperrors.WithDetails(map[string]any{
"contractVersion": "doc.operation.v1", "status": "partial_success", "nodeId": nodeID,
"resourceId": resourceID, "resourceUrl": resourceURL, "fileName": fileName, "verified": false,
"steps": []map[string]any{
{"name": "resolve_upload", "status": "success"},
{"name": "upload_oss", "status": "success"},
{"name": "insert_block", "status": "success"},
{"name": "verify", "status": "failed"},
},
}),
apperrors.WithCause(cause),
)
}
var docMediaVerifyWait = waitForDocVerification
func verifyInsertedDocMedia(ctx context.Context, nodeID, blockID, resourceID, resourceURL string) (string, error) {
delays := []time.Duration{250 * time.Millisecond, 500 * time.Millisecond, time.Second, 2 * time.Second, 4 * time.Second, 8 * time.Second}
var lastErr error
for attempt := 0; attempt <= len(delays); attempt++ {
blocks, err := readAllDocBlocksForVerification(ctx, nodeID)
if err != nil {
lastErr = err
} else {
lastErr = nil
if found := findVerifiedMediaBlock(blocks, blockID, resourceID, resourceURL); found != "" {
return found, nil
}
}
if attempt < len(delays) {
if err := docMediaVerifyWait(ctx, delays[attempt]); err != nil {
return "", err
}
}
}
if lastErr != nil {
return "", fmt.Errorf("媒体资源在有界回读窗口内仍无法读取: %w", lastErr)
}
return "", fmt.Errorf("媒体资源在有界回读窗口内仍不可见")
}
func waitForDocVerification(ctx context.Context, delay time.Duration) error {
if ctx == nil {
ctx = context.Background()
}
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-timer.C:
return nil
}
}
func readAllDocBlocksForVerification(ctx context.Context, nodeID string) ([]any, error) {
const pageSize = 50
const maxItems = 5000
all := make([]any, 0, pageSize)
seenPageIdentities := map[string]bool{}
for start := 0; start < maxItems; start += pageSize {
text, err := callMCPToolReturnTextOnServer(ctx, "doc", "list_document_blocks", map[string]any{
"nodeId": nodeID, "format": "jsonml", "startIndex": start, "endIndex": start + pageSize - 1,
})
if err != nil {
return nil, err
}
var payload map[string]any
if err := json.Unmarshal([]byte(text), &payload); err != nil {
return nil, fmt.Errorf("解析 list_document_blocks 回读失败: %w", err)
}
payload = nestedDocMap(payload)
blocks, ok := payload["blocks"].([]any)
if !ok {
return nil, fmt.Errorf("list_document_blocks 回读缺少 blocks 数组")
}
pageIdentity := docBlockPageIdentity(blocks)
if pageIdentity != "" && seenPageIdentities[pageIdentity] {
return nil, fmt.Errorf("list_document_blocks 分页停滞")
}
if pageIdentity != "" {
seenPageIdentities[pageIdentity] = true
}
all = append(all, blocks...)
hasMore, hasMoreKnown := payload["hasMore"].(bool)
if hasMoreKnown && !hasMore {
return all, nil
}
if !hasMoreKnown {
if total, ok := docNumberAsInt(payload["totalCount"]); ok && len(all) >= total {
return all, nil
}
}
if !hasMoreKnown && len(blocks) < pageSize {
return all, nil
}
if len(blocks) == 0 {
return nil, fmt.Errorf("list_document_blocks 声明仍有下一页但当前页为空")
}
}
return nil, fmt.Errorf("文档块超过安全回读上限")
}
func docBlockPageIdentity(blocks []any) string {
if len(blocks) == 0 {
return ""
}
ids := make([]string, 0, len(blocks))
for _, value := range blocks {
id := ""
switch block := value.(type) {
case map[string]any:
id = directDocBlockIdentity(block)
if id == "" {
if element, ok := block["element"].(map[string]any); ok {
id = directDocBlockIdentity(element)
}
}
case []any:
if len(block) > 1 {
if attributes, ok := block[1].(map[string]any); ok {
id = directDocBlockIdentity(attributes)
}
}
}
if id == "" {
return ""
}
ids = append(ids, id)
}
encoded, _ := json.Marshal(ids)
return string(encoded)
}
func directDocBlockIdentity(block map[string]any) string {
for _, key := range []string{"blockId", "id", "uuid", "elementId"} {
if text, ok := block[key].(string); ok && strings.TrimSpace(text) != "" {
return strings.TrimSpace(text)
}
}
return ""
}
func nestedDocMap(data map[string]any) map[string]any {
for _, key := range []string{"result", "data"} {
if nested, ok := data[key].(map[string]any); ok {
return nestedDocMap(nested)
}
}
return data
}
func nestedDocString(value any, keys ...string) string {
switch typed := value.(type) {
case map[string]any:
for _, key := range keys {
if text, ok := typed[key].(string); ok && strings.TrimSpace(text) != "" {
return strings.TrimSpace(text)
}
}
orderedKeys := make([]string, 0, len(typed))
for key := range typed {
orderedKeys = append(orderedKeys, key)
}
sort.Strings(orderedKeys)
for _, key := range orderedKeys {
if text := nestedDocString(typed[key], keys...); text != "" {
return text
}
}
case []any:
for _, child := range typed {
if text := nestedDocString(child, keys...); text != "" {
return text
}
}
}
return ""
}
// insertedDocBlockID only accepts explicit block IDs from the insert result or
// known response wrappers. Arbitrary IDs may belong to the document, operator,
// or request and must not become a hard constraint for the media readback.
func insertedDocBlockID(data map[string]any) string {
for _, key := range []string{"blockId", "elementId"} {
if text, ok := data[key].(string); ok && strings.TrimSpace(text) != "" {
return strings.TrimSpace(text)
}
}
for _, wrapper := range []string{"result", "data", "content"} {
if inner, ok := data[wrapper].(map[string]any); ok {
if text := insertedDocBlockID(inner); text != "" {
return text
}
}
}
return ""
}
func findVerifiedMediaBlock(blocks []any, blockID, resourceID, resourceURL string) string {
for _, value := range blocks {
candidateID := nestedDocString(value, "blockId", "id", "uuid")
if candidateID == "" || (blockID != "" && candidateID != blockID) {
continue
}
mediaValue := docMediaReadbackValue(value)
if resourceID != "" && nestedDocString(mediaValue, "resourceId") == resourceID {
return candidateID
}
if resourceURL != "" && nestedDocString(mediaValue, "resourceUrl", "src") == resourceURL {
return candidateID
}
}
return ""
}
func docMediaReadbackValue(value any) any {
block, ok := value.(map[string]any)
if !ok {
return value
}
encoded, ok := block["jsonml"].(string)
if !ok || strings.TrimSpace(encoded) == "" {
return value
}
var decoded any
if json.Unmarshal([]byte(encoded), &decoded) != nil {
return value
}
return decoded
}
func docNumberAsInt(value any) (int, bool) {
switch typed := value.(type) {
case float64:
if typed >= 0 && typed == float64(int(typed)) {
return int(typed), true
}
case int:
return typed, typed >= 0
}
return 0, false
}
// parseAttachmentUploadInfo extracts uploadUrl, resourceId and resourceUrl from the MCP tool response.
func parseAttachmentUploadInfo(text string) (uploadURL, resourceID, resourceURL string, err error) {
var data map[string]any
+226 -1
View File
@@ -3,7 +3,9 @@ package helpers
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
@@ -13,6 +15,7 @@ import (
"testing"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/pkg/edition"
"github.com/spf13/cobra"
)
@@ -74,6 +77,8 @@ func TestCrossPlatformCoverageDocUploadAndMediaErrorEdges(t *testing.T) {
t.Cleanup(func() { os.Args = oldArgs })
oldPut, oldGet := httpPutFile, httpGetFile
t.Cleanup(func() { httpPutFile, httpGetFile = oldPut, oldGet })
testseam.Swap(t, &helperSleep, func(time.Duration) {})
testseam.Swap(t, &docMediaVerifyWait, func(context.Context, time.Duration) error { return nil })
file := filepath.Join(t.TempDir(), "file.txt")
if err := os.WriteFile(file, []byte("content"), 0o600); err != nil {
t.Fatal(err)
@@ -193,6 +198,47 @@ func TestCrossPlatformCoverageDocUploadAndMediaErrorEdges(t *testing.T) {
t.Fatal("media insert failure returned nil")
}
})
t.Run("media insert response parse failure", func(t *testing.T) {
httpPutFile = func(context.Context, string, map[string]string, string, int64) error { return nil }
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{text: `{"uploadUrl":"https://upload","resourceId":"resource"}`},
{text: `{`},
}}
if err := mediaCommand(t, caller, file, "text/plain"); err == nil {
t.Fatal("invalid media insert response returned nil")
}
})
t.Run("small image verifies src readback when upload also returns resource ID", func(t *testing.T) {
httpPutFile = func(context.Context, string, map[string]string, string, int64) error { return nil }
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{text: `{"uploadUrl":"https://upload","resourceId":"resource","resourceUrl":"https://image"}`},
{text: `{"blockId":"media-block"}`},
{text: `{"blocks":[{"blockId":"media-block","jsonml":"[\"p\",{\"uuid\":\"media-block\"},[\"img\",{\"src\":\"https://image\"}]]"}],"hasMore":false}`},
}}
if err := mediaCommand(t, caller, file, "image/png"); err != nil {
t.Fatal(err)
}
if caller.calls != 3 {
t.Fatalf("media insert calls = %d, want credential, insert, and one readback", caller.calls)
}
if caller.args["format"] != "jsonml" {
t.Fatalf("media readback format = %#v, want jsonml", caller.args["format"])
}
})
t.Run("unrelated insert response IDs do not constrain media readback", func(t *testing.T) {
httpPutFile = func(context.Context, string, map[string]string, string, int64) error { return nil }
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{text: `{"uploadUrl":"https://upload","resourceId":"resource","resourceUrl":"https://image"}`},
{text: `{"id":"document-id","operator":{"id":"operator-id"},"data":{"request":{"id":"request-id"}}}`},
{text: `{"blocks":[{"blockId":"media-block","jsonml":"[\"p\",{\"uuid\":\"media-block\"},[\"img\",{\"src\":\"https://image\"}]]"}],"hasMore":false}`},
}}
if err := mediaCommand(t, caller, file, "image/png"); err != nil {
t.Fatal(err)
}
if caller.calls != 3 {
t.Fatalf("media insert calls = %d, want credential, insert, and one readback", caller.calls)
}
})
t.Run("large image becomes attachment", func(t *testing.T) {
httpPutFile = func(context.Context, string, map[string]string, string, int64) error { return nil }
large := filepath.Join(t.TempDir(), "large.png")
@@ -202,7 +248,11 @@ func TestCrossPlatformCoverageDocUploadAndMediaErrorEdges(t *testing.T) {
if err := os.Truncate(large, 21*1024*1024); err != nil {
t.Fatal(err)
}
caller := &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"uploadUrl":"https://upload","resourceId":"resource","resourceUrl":"https://image"}`}, {text: `{}`}}}
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{text: `{"uploadUrl":"https://upload","resourceId":"resource","resourceUrl":"https://image"}`},
{text: `{}`},
{text: `{"blocks":[{"blockId":"media-block","element":{"attachment":{"resourceId":"resource"}}}],"hasMore":false}`},
}}
if err := mediaCommand(t, caller, large, "image/png"); err != nil {
t.Fatal(err)
}
@@ -240,6 +290,181 @@ func TestCrossPlatformCoverageDefaultDocHTTPTransportEdges(t *testing.T) {
}
}
func TestCrossPlatformCoverageDocMediaReadbackDefensiveEdges(t *testing.T) {
testseam.Swap(t, &helperSleep, func(time.Duration) {})
testseam.Swap(t, &docMediaVerifyWait, func(context.Context, time.Duration) error { return nil })
ctx := context.Background()
for _, tc := range []struct {
name string
steps []scriptedToolStep
}{
{"call failure", []scriptedToolStep{{err: errors.New("read")}}},
{"invalid json", []scriptedToolStep{{text: `{`}}},
{"missing blocks", []scriptedToolStep{{text: `{}`}}},
{"stalled page", []scriptedToolStep{{text: `{"blocks":[{"id":"a"}],"hasMore":true}`}, {text: `{"blocks":[{"id":"a"}],"hasMore":true}`}}},
{"empty continued page", []scriptedToolStep{{text: `{"blocks":[],"hasMore":true}`}}},
} {
t.Run(tc.name, func(t *testing.T) {
installScriptedCaller(t, &scriptedToolCaller{steps: tc.steps})
if _, err := readAllDocBlocksForVerification(ctx, "node"); err == nil {
t.Fatal("defensive readback returned nil")
}
})
}
t.Run("total count and nested payload", func(t *testing.T) {
installScriptedCaller(t, &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"data":{"blocks":[{"id":"a"}],"totalCount":1}}`}}})
blocks, err := readAllDocBlocksForVerification(ctx, "node")
if err != nil || len(blocks) != 1 {
t.Fatalf("blocks=%#v err=%v", blocks, err)
}
})
t.Run("identical adjacent pages advance by requested indexes", func(t *testing.T) {
blocks := make([]any, 50)
for index := range blocks {
blocks[index] = map[string]any{"blockType": "paragraph"}
}
first, err := json.Marshal(map[string]any{"blocks": blocks, "hasMore": true, "totalCount": 100})
if err != nil {
t.Fatal(err)
}
second, err := json.Marshal(map[string]any{"blocks": blocks, "hasMore": false, "totalCount": 100})
if err != nil {
t.Fatal(err)
}
caller := &scriptedToolCaller{steps: []scriptedToolStep{{text: string(first)}, {text: string(second)}}}
installScriptedCaller(t, caller)
got, err := readAllDocBlocksForVerification(ctx, "node")
if err != nil || len(got) != 100 || caller.calls != 2 {
t.Fatalf("blocks=%d calls=%d err=%v, want 100 blocks from two pages", len(got), caller.calls, err)
}
})
t.Run("explicit has more overrides inconsistent total count", func(t *testing.T) {
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{text: `{"blocks":[{"id":"first"}],"hasMore":true,"totalCount":1}`},
{text: `{"blocks":[{"id":"second"}],"hasMore":false,"totalCount":2}`},
}}
installScriptedCaller(t, caller)
got, err := readAllDocBlocksForVerification(ctx, "node")
if err != nil || len(got) != 2 || caller.calls != 2 {
t.Fatalf("blocks=%d calls=%d err=%v, want both explicitly advertised pages", len(got), caller.calls, err)
}
})
t.Run("page identity accepts nested and JSONML block IDs", func(t *testing.T) {
if got := docBlockPageIdentity([]any{map[string]any{"element": map[string]any{"blockId": "nested-block"}}}); got != `["nested-block"]` {
t.Fatalf("nested block page identity = %q", got)
}
if got := docBlockPageIdentity([]any{[]any{"p", map[string]any{"uuid": "jsonml-block"}}}); got != `["jsonml-block"]` {
t.Fatalf("JSONML block page identity = %q", got)
}
})
t.Run("unknown pagination short page", func(t *testing.T) {
installScriptedCaller(t, &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"blocks":[{"id":"a"}]}`}}})
if blocks, err := readAllDocBlocksForVerification(ctx, "node"); err != nil || len(blocks) != 1 {
t.Fatalf("blocks=%#v err=%v", blocks, err)
}
})
t.Run("bounded retry failure", func(t *testing.T) {
installScriptedCaller(t, &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"blocks":[],"hasMore":false}`}}})
if _, err := verifyInsertedDocMedia(ctx, "node", "", "missing", ""); err == nil {
t.Fatal("missing media unexpectedly verified")
}
})
t.Run("transient read failure retries", func(t *testing.T) {
caller := &scriptedToolCaller{steps: []scriptedToolStep{
{err: errors.New("temporary read failure")},
{text: `{"blocks":[{"blockId":"media-block","element":{"attachment":{"resourceId":"resource"}}}],"hasMore":false}`},
}}
installScriptedCaller(t, caller)
blockID, err := verifyInsertedDocMedia(ctx, "node", "media-block", "resource", "")
if err != nil || blockID != "media-block" || caller.calls != 2 {
t.Fatalf("blockID=%q calls=%d err=%v", blockID, caller.calls, err)
}
})
t.Run("block identity alone does not verify media", func(t *testing.T) {
installScriptedCaller(t, &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"blocks":[{"blockId":"media-block","element":{"attachment":{"resourceId":"other-resource"}}}],"hasMore":false}`}}})
if _, err := verifyInsertedDocMedia(ctx, "node", "media-block", "resource", ""); err == nil {
t.Fatal("matching block ID with a different resource unexpectedly verified")
}
})
t.Run("block read safety limit", func(t *testing.T) {
steps := make([]scriptedToolStep, 100)
for index := range steps {
steps[index] = scriptedToolStep{text: fmt.Sprintf(`{"blocks":[{"id":"block-%d"}],"hasMore":true}`, index)}
}
installScriptedCaller(t, &scriptedToolCaller{steps: steps})
if _, err := readAllDocBlocksForVerification(ctx, "node"); err == nil {
t.Fatal("oversized block read returned nil")
}
})
if got := nestedDocMap(map[string]any{"result": map[string]any{"data": map[string]any{"ok": true}}}); got["ok"] != true {
t.Fatalf("nested map=%#v", got)
}
if nestedDocString(map[string]any{"x": []any{map[string]any{"id": " nested "}}}, "id") != "nested" || nestedDocString(3, "id") != "" {
t.Fatal("nested string traversal failed")
}
if got := nestedDocString(map[string]any{"z": map[string]any{"id": "last"}, "a": map[string]any{"id": "first"}}, "id"); got != "first" {
t.Fatalf("nested string traversal = %q, want deterministic key order", got)
}
if got := insertedDocBlockID(map[string]any{"id": "document", "result": map[string]any{"data": map[string]any{"blockId": " block "}}}); got != "block" {
t.Fatalf("trusted inserted block ID = %q, want block", got)
}
if got := insertedDocBlockID(map[string]any{"id": "document", "operator": map[string]any{"id": "operator"}, "data": map[string]any{"request": map[string]any{"id": "request"}}}); got != "" {
t.Fatalf("untrusted inserted block ID = %q, want empty", got)
}
blocks := []any{map[string]any{"id": "block", "resourceId": "rid"}, map[string]any{"id": "url-block", "resourceUrl": "https://media"}, map[string]any{"id": "src-block", "jsonml": `["p",{},["img",{"src":"https://image"}]]`}}
if findVerifiedMediaBlock(blocks, "block", "rid", "") != "block" || findVerifiedMediaBlock(blocks, "", "rid", "") != "block" || findVerifiedMediaBlock(blocks, "", "", "https://media") != "url-block" || findVerifiedMediaBlock(blocks, "src-block", "upload-resource", "https://image") != "src-block" || findVerifiedMediaBlock(blocks, "other-block", "upload-resource", "https://image") != "" || findVerifiedMediaBlock(blocks, "block", "wrong", "") != "" || findVerifiedMediaBlock(blocks, "", "missing", "") != "" {
t.Fatal("media block matching failed")
}
if findVerifiedMediaBlock([]any{map[string]any{"id": "bad-jsonml", "jsonml": `[`}}, "bad-jsonml", "", "https://image") != "" {
t.Fatal("invalid JSONML unexpectedly verified")
}
if got := docMediaReadbackValue("plain"); got != "plain" {
t.Fatalf("non-object media readback = %#v, want unchanged value", got)
}
for _, tc := range []struct {
value any
want bool
}{
{float64(3), true}, {float64(-1), false}, {1.5, false}, {3, true}, {-1, false}, {"3", false},
} {
_, ok := docNumberAsInt(tc.value)
if ok != tc.want {
t.Fatalf("docNumberAsInt(%#v) ok=%v want=%v", tc.value, ok, tc.want)
}
}
}
func TestCrossPlatformCoverageDocMediaReadbackStopsOnCancellation(t *testing.T) {
if err := waitForDocVerification(nil, time.Nanosecond); err != nil {
t.Fatalf("completed verification wait = %v", err)
}
cancelled, cancel := context.WithCancel(context.Background())
cancel()
if err := waitForDocVerification(cancelled, time.Hour); !errors.Is(err, context.Canceled) {
t.Fatalf("cancelled verification wait = %v, want context.Canceled", err)
}
caller := &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"blocks":[],"hasMore":false}`}}}
installScriptedCaller(t, caller)
if _, err := verifyInsertedDocMedia(cancelled, "node", "", "missing", ""); !errors.Is(err, context.Canceled) {
t.Fatalf("cancelled media verification = %v, want context.Canceled", err)
}
if caller.calls != 1 {
t.Fatalf("cancelled media verification calls = %d, want 1", caller.calls)
}
}
func TestCrossPlatformCoverageDocCreateUpdateAndBlockCommandEdges(t *testing.T) {
oldDeps, oldArgs, oldPut, oldGet := deps, os.Args, httpPutFile, httpGetFile
t.Cleanup(func() {
@@ -9,6 +9,7 @@ import (
"path/filepath"
"strings"
"testing"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
"github.com/spf13/cobra"
@@ -194,6 +195,7 @@ func TestCrossPlatformCoverageDocDeprecationWrappersCoverage(t *testing.T) {
}
func TestCrossPlatformCoverageRunDocUploadDownloadAndMediaCoverage(t *testing.T) {
testseam.Swap(t, &docMediaVerifyWait, func(context.Context, time.Duration) error { return nil })
oldArgs := os.Args
os.Args = []string{"dws", "doc"}
t.Cleanup(func() { os.Args = oldArgs })
+45 -1
View File
@@ -60,6 +60,11 @@ func translateBatchOp(op map[string]any) (map[string]any, error) {
if !ok {
return nil, fmt.Errorf("unsupported toolName %q: must be a CLI command name (e.g. \"range clear\", \"range update\", \"merge-cells\"). Run 'dws sheet batch-update --help' for the full list", toolName)
}
if toolName == "set-dropdown" {
if err := validateBatchSetDropdownInput(input); err != nil {
return nil, err
}
}
return map[string]any{
"toolName": mapping.mcpTool,
@@ -67,6 +72,33 @@ func translateBatchOp(op map[string]any) (map[string]any, error) {
}, nil
}
func validateBatchSetDropdownInput(input map[string]any) error {
if _, exists := input["source-colors"]; exists {
return fmt.Errorf("set-dropdown: 顶层 source-colors 不受支持;Inline 颜色请写入 options[].color,SourceRange 颜色写入暂不支持")
}
if _, exists := input["colors"]; exists {
return fmt.Errorf("set-dropdown: 顶层 colors 不受支持;Inline 颜色请写入 options[].color,SourceRange 颜色写入暂不支持")
}
options, hasOptions := input["options"]
hasOptions = hasOptions && options != nil
sourceRange := batchStr(input, "source-range")
sourceSheetID := batchStr(input, "source-sheet-id")
hasSourceRange := sourceRange != ""
if hasOptions == hasSourceRange {
return fmt.Errorf("set-dropdown: options 与 source-range 必须且只能指定一个")
}
if hasSourceRange != (sourceSheetID != "") {
return fmt.Errorf("set-dropdown: source-range 与 source-sheet-id 必须同时指定")
}
if hasSourceRange {
if err := validateDropdownSourceRangeInput(sourceSheetID, sourceRange); err != nil {
return fmt.Errorf("set-dropdown: %w", err)
}
}
return nil
}
// ── BuildXxxArgs: CLI flag → MCP param 转换函数 ──────────────────────────────────
// 每个函数接收 CLI flag 名(kebab-case)的 map,输出 MCP 参数名(camelCase)的 map。
// 目前集中放在此文件;后续拆分命令文件时可移到各命令所在文件。
@@ -215,7 +247,15 @@ func BuildSetDropdownArgs(input map[string]any) map[string]any {
args := map[string]any{
"sheetId": batchStr(input, "sheet-id"),
"range": batchStr(input, "range"),
"options": input["options"],
}
if options, ok := input["options"]; ok && options != nil {
args["options"] = options
}
if sourceRange := batchStr(input, "source-range"); sourceRange != "" {
args["sourceRange"] = map[string]any{
"sheetId": batchStr(input, "source-sheet-id"),
"a1Notation": sourceRange,
}
}
if v, ok := input["multi-select"]; ok {
args["enableMultiSelect"] = v
@@ -417,6 +457,8 @@ func newBatchUpdateCmd() *cobra.Command {
toolName 使用 CLI 命令名(与原子命令一致),input 的键用 CLI flag 名去掉 --。
CLI 层自动翻译为 MCP toolName + 参数名,无需记忆 MCP 参数名。
source-range 的语义按 toolName 隔离:set-dropdown 中表示下拉候选项来源,
range fill/copy-to/move-to 中表示待填充、复制或移动的数据源区域。
支持的 CLI 命令名:
range clear / range update / merge-cells / unmerge-cells / update-dimension
@@ -425,6 +467,8 @@ CLI 层自动翻译为 MCP toolName + 参数名,无需记忆 MCP 参数名。
set-dropdown / delete-dropdown / csv-put / delete-float-image
其中 csv-put 与独立命令语义一致:CSV 字段值以 = 开头时按公式解析,
前加单引号(例如 "'=1+1")时写入以 = 开头的字面文本。
set-dropdown 不接受顶层 colors/source-colors;Inline 颜色应写在 options[].color,
SourceRange 颜色写入暂不支持。
注意:batch-update 中 group-dimension 适合默认展开分组;需要 --group-state fold 时请使用独立
dws sheet group-dimension 命令。
+9 -1
View File
@@ -23,7 +23,15 @@ func TestCrossPlatformCoverageSheetBatchOperationTranslationCoversEveryMapping(t
"group-state": "fold",
}
for name, mapping := range batchOpDispatch {
got, err := translateBatchOp(map[string]any{"toolName": name, "input": input})
opInput := input
if name == "set-dropdown" {
opInput = make(map[string]any, len(input))
for key, value := range input {
opInput[key] = value
}
delete(opInput, "source-range")
}
got, err := translateBatchOp(map[string]any{"toolName": name, "input": opInput})
if err != nil {
t.Errorf("translateBatchOp(%q): %v", name, err)
continue
+10 -2
View File
@@ -388,7 +388,10 @@ range update 与合并区域冲突时返回 MERGED_CELLS_CONFLICT 的行为。
csv 带 [row=N] 行号前缀的 CSV 文本
colIndices 列字母映射数组(定位列用 colIndices[j],禁止手数逗号)
rowIndices 行号映射数组
hasMore 是否因 maxChars 截断
hasMore 目标范围是否还有未返回数据
truncationReasons 部分返回原因:max_cells / max_chars
resolvedRange 未传 --range 时底层解析出的完整目标范围
returnedRange 本次实际完整返回的范围
取值模式(--value-render-option):
formatted_value 格式化后的展示值(默认),如 ¥1,000.00、2025-06-01
@@ -398,9 +401,14 @@ range update 与合并区域冲突时返回 MERGED_CELLS_CONFLICT 的行为。
与 range read 的区别:
- CSV 格式 token 消耗约为 JSON 的 1/3
- 支持选择取值模式
- 自动防爆(max-chars 截断 + has_more 标志)
- 自动防爆(30,000 单元格 / max-chars 上限 + hasMore 标志)
- [row=N] 前缀防止行号计算错误
当 hasMore=true 时本命令不会自动续读;请结合目标范围和
returnedRange 从下一行显式传 --range。max_cells 不能通过增大
--max-chars 解决。收到 forbidden.document.sizeOverLimit 表示工作簿整体
无法装载,应创建更小副本或拆分工作簿,缩小 --range 不能解决。
注意:csv-get 不返回合并单元格结构。查看合并范围请使用
dws sheet info --node NODE_ID --sheet-id SHEET_ID --format json,并读取 mergedRanges。`,
Example: ` dws sheet csv-get --node NODE_ID
+93 -22
View File
@@ -6,6 +6,7 @@ import (
"strconv"
"strings"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cli"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/corecmd/contract"
"github.com/spf13/cobra"
)
@@ -18,6 +19,23 @@ func sizeTypeEnumHint(dimension string) string {
return "pixel / standard"
}
func validateDropdownSourceRangeInput(sourceSheetID, sourceRange string) error {
if strings.TrimSpace(sourceSheetID) == "" {
return fmt.Errorf("--source-sheet-id 不能为空")
}
sourceRange = strings.TrimSpace(sourceRange)
if sourceRange == "" {
return fmt.Errorf("--source-range 不能为空")
}
if strings.Contains(sourceRange, "!") {
return fmt.Errorf("--source-range 不能包含工作表前缀;请通过 --source-sheet-id 指定来源工作表")
}
if strings.HasPrefix(sourceRange, "=") || strings.Contains(sourceRange, ",") {
return fmt.Errorf("--source-range 必须是单一连续区域,不能是公式、表达式或多区域引用")
}
return nil
}
// newDimensionCmds creates dimension-related commands: insert/delete/update/move/add-dimension,
// merge-cells, unmerge-cells, and dropdown commands (set/get/delete-dropdown).
func newDimensionCmds() []*cobra.Command {
@@ -775,15 +793,24 @@ sheetId 支持传入工作表 ID 或工作表名称,可通过 sheet list 获
Short: "设置下拉列表",
Long: `在钉钉表格的指定单元格范围内设置下拉列表。
设置后,用户可以在这些单元格中从预定义的选项列表中选择值。
支持自定义每个选项的颜色和是否允许多选。
Inline 模式通过 --options 提供静态选项,支持选项颜色。
SourceRange 模式通过 --source-sheet-id + --source-range 引用同一工作簿内的来源区域,
支持跨工作表、整行和整列引用;读取配置时不会展开来源区域的当前值或颜色。
两种模式都支持 --multi-select,并且 --options 与 --source-range 必须且只能指定一个。
如果目标范围已存在下拉列表,会被新的配置覆盖。
nodeId 支持传入文档链接 URL 或文档 ID(dentryUuid),系统自动识别。
sheetId 支持传入工作表 ID 或工作表名称,可通过 sheet list 获取。
--options 为 JSON 数组,每个元素包含 value(必填)和 color(可选)。
选项值不能包含英文逗号。`,
选项值不能包含英文逗号。
--source-range 只接受不含工作表前缀的 A1 区域,例如 T1:T3、T:T 或 1:3;
来源工作表由 --source-sheet-id 单独指定。SourceRange 颜色写入暂不支持。
已验证行为:工作表重命名、在引用前插入行/列、删除引用前的行会自动调整引用并保持 valid;
已验证的 move-dimension 场景会使引用变为 invalid。列删除、删除整个来源区域或来源工作表等未覆盖场景不能预设结果;
结构操作后应先回读 sourceRangeStatus,只有 invalid 时才重新选择来源并写入。`,
Example: ` # 设置单选下拉列表
dws sheet set-dropdown --node NODE_ID --sheet-id SHEET_ID --range "A2:A100" \
--options '[{"value":"选项1"},{"value":"选项2"},{"value":"选项3"}]'
@@ -793,32 +820,63 @@ sheetId 支持传入工作表 ID 或工作表名称,可通过 sheet list 获
--options '[{"value":"高","color":"#ff0000"},{"value":"中","color":"#ffaa00"},{"value":"低","color":"#00ff00"}]' \
--multi-select
# 引用同一工作簿内另一工作表的区域作为候选项来源
dws sheet set-dropdown --node NODE_ID --sheet-id TARGET_SHEET_ID --range "C2:C100" \
--source-sheet-id SOURCE_SHEET_ID --source-range "T1:T3"
# 使用文档链接 URL
dws sheet set-dropdown --node "https://alidocs.dingtalk.com/i/nodes/<DOC_UUID>" \
--sheet-id SHEET_ID --range "C1:C10" --options '[{"value":"是"},{"value":"否"}]'`,
RunE: func(cmd *cobra.Command, args []string) error {
if err := validateRequiredFlags(cmd, "node", "sheet-id", "range"); err != nil {
return err
}
optionsStr := mustGetFlag(cmd, "options")
var options []map[string]any
if err := json.Unmarshal([]byte(optionsStr), &options); err != nil {
return fmt.Errorf("--options JSON 解析失败: %w", err)
sourceSheetID := mustGetFlag(cmd, "source-sheet-id")
sourceRange := mustGetFlag(cmd, "source-range")
hasOptions := optionsStr != ""
hasSourceRange := sourceRange != ""
if hasOptions == hasSourceRange {
return fmt.Errorf("--options 与 --source-range 必须且只能指定一个")
}
if len(options) == 0 {
return fmt.Errorf("--options 至少包含 1 个选项")
if hasSourceRange && sourceSheetID == "" {
return fmt.Errorf("使用 --source-range 时必须同时指定 --source-sheet-id")
}
for i, opt := range options {
val, ok := opt["value"].(string)
if !ok || val == "" {
return fmt.Errorf("--options[%d] 缺少必填的 value 字段或 value 为空", i)
}
if strings.Contains(val, ",") {
return fmt.Errorf("--options[%d].value 不能包含英文逗号: %q", i, val)
}
if !hasSourceRange && sourceSheetID != "" {
return fmt.Errorf("使用 --source-sheet-id 时必须同时指定 --source-range")
}
toolArgs := map[string]any{
"nodeId": mustGetFlag(cmd, "node"),
"sheetId": mustGetFlag(cmd, "sheet-id"),
"range": mustGetFlag(cmd, "range"),
"options": options,
}
if hasOptions {
var options []map[string]any
if err := json.Unmarshal([]byte(optionsStr), &options); err != nil {
return fmt.Errorf("--options JSON 解析失败: %w", err)
}
if len(options) == 0 {
return fmt.Errorf("--options 至少包含 1 个选项")
}
for i, opt := range options {
val, ok := opt["value"].(string)
if !ok || val == "" {
return fmt.Errorf("--options[%d] 缺少必填的 value 字段或 value 为空", i)
}
if strings.Contains(val, ",") {
return fmt.Errorf("--options[%d].value 不能包含英文逗号: %q", i, val)
}
}
toolArgs["options"] = options
} else {
if err := validateDropdownSourceRangeInput(sourceSheetID, sourceRange); err != nil {
return err
}
toolArgs["sourceRange"] = map[string]any{
"sheetId": sourceSheetID,
"a1Notation": sourceRange,
}
}
if multiSelect, _ := cmd.Flags().GetBool("multi-select"); multiSelect {
toolArgs["enableMultiSelect"] = true
@@ -839,36 +897,49 @@ sheetId 支持传入工作表 ID 或工作表名称,可通过 sheet list 获
CLIPath: "sheet set-dropdown",
PrimaryCLIPath: "sheet set-dropdown",
},
Description: "为指定范围设置下拉列表(可多选、可带颜色)。",
Description: "为指定范围设置 Inline 或 SourceRange 下拉列表(可多选;颜色仅 Inline 支持)。",
Interface: &contract.InterfaceSpec{
Mode: "mcp",
Availability: "available",
Ref: &contract.InterfaceRefSpec{ProductID: "sheet", RPCName: "set_dropdown_lists"},
},
Selection: contract.SelectionSpec{
AgentSummary: "为指定范围设置下拉列表(可多选、可带颜色)。",
AgentSummary: "为指定范围设置 Inline 或 SourceRange 下拉列表;SourceRange 可跨同一工作簿内的工作表。",
UseWhen: []string{"需要给单元格配置可选值下拉约束时"},
AvoidWhen: []string{"查看已有下拉用 get-dropdown;移除下拉用 delete-dropdown"},
Examples: []string{"dws sheet set-dropdown --node <NODE_ID> --sheet-id <SHEET_ID> --range \"A2:A100\" --options '[{\"value\":\"选项1\"},{\"value\":\"选项2\"}]'"},
Examples: []string{
"dws sheet set-dropdown --node <NODE_ID> --sheet-id <SHEET_ID> --range \"A2:A100\" --options '[{\"value\":\"选项1\"},{\"value\":\"选项2\"}]'",
"dws sheet set-dropdown --node <NODE_ID> --sheet-id <TARGET_SHEET_ID> --range \"B2:B100\" --source-sheet-id <SOURCE_SHEET_ID> --source-range \"T1:T3\"",
},
},
Parameters: []contract.ParamDecl{
{Name: "multi-select", Property: "enableMultiSelect"},
{Name: "node", Property: "nodeId"},
{Name: "source-sheet-id", Property: "sourceRange.sheetId", Required: boolPtr(false), RequiredWhen: "--source-range is provided"},
{Name: "source-range", Property: "sourceRange.a1Notation", Required: boolPtr(false), RequiredWhen: "exactly one of --options or --source-range must be provided"},
},
},
})
setDropdownCmd.Flags().String("node", "", "表格文档 ID 或 URL (必填)")
setDropdownCmd.Flags().String("sheet-id", "", "工作表 ID 或名称 (必填)")
setDropdownCmd.Flags().String("range", "", "目标单元格范围,A1 表示法,如 A2:A100 (必填)")
setDropdownCmd.Flags().String("options", "", `下拉选项 JSON 数组 (必填),如 '[{"value":"选项1","color":"#ff0000"}]'`)
setDropdownCmd.Flags().String("options", "", `Inline 下拉选项 JSON 数组,与 --source-range 二选一,如 '[{"value":"选项1","color":"#ff0000"}]'`)
setDropdownCmd.Flags().String("source-sheet-id", "", "SourceRange 来源工作表 ID;使用 --source-range 时必填")
setDropdownCmd.Flags().String("source-range", "", "SourceRange 来源区域,与 --options 二选一;不含工作表前缀,如 T1:T3、T:T 或 1:3")
setDropdownCmd.Flags().Bool("multi-select", false, "是否允许多选(默认单选)")
cli.AnnotateRuntimeFlagFormat(setDropdownCmd, "source-range", "a1-range")
setDropdownCmd.MarkFlagsOneRequired("options", "source-range")
setDropdownCmd.MarkFlagsMutuallyExclusive("options", "source-range")
setDropdownCmd.MarkFlagsRequiredTogether("source-sheet-id", "source-range")
getDropdownCmd := &cobra.Command{
Use: "get-dropdown",
Short: "获取下拉列表配置",
Long: `查询钉钉表格指定范围内的下拉列表配置。
返回范围内所有单元格的下拉列表配置信息,包括选项值和颜色。
Inline 配置返回 conditionValues 和 options;SourceRange 配置返回
sourceType/sourceRangeStatus/enableMultiSelect,且仅在 sourceRangeStatus="valid" 时返回
sourceRange。invalid 结果仍保留配置组,但省略 sourceRange;两种状态都不展开来源区域的候选值或颜色。
如果范围内存在多个不同的下拉列表配置,会分别返回每组配置及其覆盖的单元格列表。
如果范围内没有设置下拉列表,返回空。
@@ -0,0 +1,248 @@
package helpers
import (
"strings"
"testing"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/corecmd/runtimeannotate"
)
func TestCrossPlatformCoverageSetDropdownSourceRangeCommand(t *testing.T) {
caller := &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"success":true}`}}}
installScriptedCaller(t, caller)
err := executeDimensionCoverage(t, "set-dropdown",
"--node", "node-1",
"--sheet-id", "target-sheet",
"--range", "B2:B100",
"--source-sheet-id", "source-sheet",
"--source-range", "$t$1:$t$3",
"--multi-select",
)
if err != nil {
t.Fatalf("set SourceRange dropdown: %v", err)
}
if caller.tool != "set_dropdown_lists" {
t.Fatalf("tool = %q, want set_dropdown_lists", caller.tool)
}
if _, exists := caller.args["options"]; exists {
t.Fatalf("SourceRange request must omit options: %#v", caller.args)
}
source, ok := caller.args["sourceRange"].(map[string]any)
if !ok {
t.Fatalf("sourceRange = %#v", caller.args["sourceRange"])
}
if source["sheetId"] != "source-sheet" || source["a1Notation"] != "$t$1:$t$3" {
t.Fatalf("sourceRange = %#v", source)
}
if caller.args["enableMultiSelect"] != true {
t.Fatalf("enableMultiSelect = %#v", caller.args["enableMultiSelect"])
}
}
func TestCrossPlatformCoverageSetDropdownInlineCommandStillOmitsSourceRange(t *testing.T) {
caller := &scriptedToolCaller{steps: []scriptedToolStep{{text: `{"success":true}`}}}
installScriptedCaller(t, caller)
err := executeDimensionCoverage(t, "set-dropdown",
"--node", "node-1",
"--sheet-id", "sheet-1",
"--range", "A1:A3",
"--options", `[{"value":"one","color":"#ff0000"}]`,
)
if err != nil {
t.Fatalf("set inline dropdown: %v", err)
}
if _, exists := caller.args["sourceRange"]; exists {
t.Fatalf("inline request must omit sourceRange: %#v", caller.args)
}
options, ok := caller.args["options"].([]map[string]any)
if !ok || len(options) != 1 || options[0]["value"] != "one" {
t.Fatalf("options = %#v", caller.args["options"])
}
}
func TestCrossPlatformCoverageSetDropdownModeConstraints(t *testing.T) {
base := []string{"--node", "node", "--sheet-id", "sheet", "--range", "A1"}
tests := []struct {
name string
args []string
want string
}{
{name: "neither", want: "one of the flags"},
{name: "both", args: []string{"--options", `[{"value":"one"}]`, "--source-sheet-id", "source", "--source-range", "A1:A3"}, want: "none of the others"},
{name: "source sheet without range", args: []string{"--options", `[{"value":"one"}]`, "--source-sheet-id", "source"}, want: "must all be set"},
{name: "range without source sheet", args: []string{"--source-range", "A1:A3"}, want: "must all be set"},
{name: "sheet prefix", args: []string{"--source-sheet-id", "source", "--source-range", "Sheet2!A1:A3"}, want: "不能包含工作表前缀"},
{name: "formula", args: []string{"--source-sheet-id", "source", "--source-range", "=A1:A3"}, want: "必须是单一连续区域"},
{name: "multi region", args: []string{"--source-sheet-id", "source", "--source-range", "A1:A3,C1:C3"}, want: "必须是单一连续区域"},
{name: "blank source range", args: []string{"--source-sheet-id", "source", "--source-range", " \t "}, want: "--source-range 不能为空"},
{name: "blank source sheet", args: []string{"--source-sheet-id", " \t ", "--source-range", "A1:A3"}, want: "--source-sheet-id 不能为空"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
caller := &scriptedToolCaller{}
installScriptedCaller(t, caller)
args := append(append([]string{}, base...), tc.args...)
err := executeDimensionCoverage(t, "set-dropdown", args...)
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("error = %v, want contains %q", err, tc.want)
}
if caller.calls != 0 {
t.Fatalf("calls = %d, validation must fail before request", caller.calls)
}
})
}
}
func TestCrossPlatformCoverageSetDropdownRunEValidation(t *testing.T) {
tests := []struct {
name string
options string
source string
rangeID string
want string
}{
{name: "neither mode", want: "必须且只能指定一个"},
{name: "source range without sheet", rangeID: "A1:A3", want: "必须同时指定 --source-sheet-id"},
{name: "source sheet without range", options: `[{"value":"one"}]`, source: "source", want: "必须同时指定 --source-range"},
{name: "invalid options JSON", options: "{", want: "JSON 解析失败"},
{name: "missing option value", options: `[{}]`, want: "value 为空"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
cmd := dimensionCoverageCommand(t, "set-dropdown")
for name, value := range map[string]string{
"node": "node", "sheet-id": "sheet", "range": "A1",
"options": tc.options, "source-sheet-id": tc.source, "source-range": tc.rangeID,
} {
if err := cmd.Flags().Set(name, value); err != nil {
t.Fatalf("set --%s: %v", name, err)
}
}
err := cmd.RunE(cmd, nil)
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("error = %v, want contains %q", err, tc.want)
}
})
}
}
func TestCrossPlatformCoverageSetDropdownPreservesSchemaConstraints(t *testing.T) {
cmd := dimensionCoverageCommand(t, "set-dropdown")
if constraints := runtimeannotate.CommandConstraints(cmd); !runtimeannotate.ConstraintsEmpty(constraints) {
t.Fatalf("constraints = %#v, want no new schema-level constraints", constraints)
}
}
func TestCrossPlatformCoverageBatchSetDropdownSourceRange(t *testing.T) {
got, err := translateBatchOp(map[string]any{
"toolName": "set-dropdown",
"input": map[string]any{
"sheet-id": "target-sheet",
"range": "B2:B100",
"source-sheet-id": "source-sheet",
"source-range": "T:T",
"multi-select": true,
},
})
if err != nil {
t.Fatalf("translate SourceRange batch op: %v", err)
}
input := got["input"].(map[string]any)
if _, exists := input["options"]; exists {
t.Fatalf("SourceRange batch input must omit options: %#v", input)
}
source := input["sourceRange"].(map[string]any)
if source["sheetId"] != "source-sheet" || source["a1Notation"] != "T:T" {
t.Fatalf("sourceRange = %#v", source)
}
invalid := []map[string]any{
{},
{"options": []any{map[string]any{"value": "one"}}, "source-sheet-id": "source", "source-range": "A1:A3"},
{"source-range": "A1:A3"},
{"source-sheet-id": "source", "source-range": "A1:A3", "colors": []any{"#fff"}},
{"source-sheet-id": "source", "source-range": "A1:A3", "source-colors": []any{"#fff"}},
{"source-sheet-id": "source", "source-range": "Sheet2!A1:A3"},
}
for _, value := range invalid {
if _, err := translateBatchOp(map[string]any{"toolName": "set-dropdown", "input": value}); err == nil {
t.Errorf("invalid batch SourceRange input %#v returned nil", value)
}
}
}
func TestCrossPlatformCoverageBatchSetDropdownValidationGuidance(t *testing.T) {
tests := []struct {
name string
input map[string]any
want string
}{
{
name: "inline top-level colors",
input: map[string]any{"options": []any{map[string]any{"value": "one"}}, "colors": []any{"#fff"}},
want: "Inline 颜色请写入 options[].color",
},
{
name: "source top-level colors",
input: map[string]any{"source-sheet-id": "source", "source-range": "A1:A3", "source-colors": []any{"#fff"}},
want: "SourceRange 颜色写入暂不支持",
},
{
name: "blank source range",
input: map[string]any{"source-sheet-id": "source", "source-range": " \t "},
want: "--source-range 不能为空",
},
{
name: "blank source sheet",
input: map[string]any{"source-sheet-id": " \t ", "source-range": "A1:A3"},
want: "--source-sheet-id 不能为空",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := translateBatchOp(map[string]any{"toolName": "set-dropdown", "input": tc.input})
if err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatalf("error = %v, want contains %q", err, tc.want)
}
})
}
}
func TestCrossPlatformCoverageDropdownHelpDescribesDynamicAndInvalidContracts(t *testing.T) {
setDropdown := dimensionCoverageCommand(t, "set-dropdown")
if !strings.Contains(setDropdown.Long, "会自动调整引用并保持 valid") ||
!strings.Contains(setDropdown.Long, "只有 invalid 时才重新选择来源并写入") {
t.Fatalf("set-dropdown help does not describe verified structural behavior:\n%s", setDropdown.Long)
}
getDropdown := dimensionCoverageCommand(t, "get-dropdown")
if !strings.Contains(getDropdown.Long, `仅在 sourceRangeStatus="valid" 时返回`) ||
!strings.Contains(getDropdown.Long, "invalid 结果仍保留配置组,但省略 sourceRange") {
t.Fatalf("get-dropdown help does not describe conditional sourceRange readback:\n%s", getDropdown.Long)
}
}
func TestCrossPlatformCoverageValidateSourceRangeDataValidation(t *testing.T) {
valid := []any{
map[string]any{"type": "dropdown", "sourceRange": map[string]any{"sheetId": "source", "a1Notation": "T1:T3"}},
map[string]any{"type": "dropdown", "sourceRange": map[string]any{"sheetId": "missing-sheet", "a1Notation": "not-validated-locally"}, "enableMultiSelect": true},
}
for _, value := range valid {
if err := validateDataValidation(value, "dv"); err != nil {
t.Errorf("valid SourceRange data validation %#v: %v", value, err)
}
}
invalid := []any{
map[string]any{"type": "dropdown", "options": []any{map[string]any{"value": "one"}}, "sourceRange": map[string]any{"sheetId": "source", "a1Notation": "T1:T3"}},
map[string]any{"type": "dropdown", "sourceRange": "T1:T3"},
map[string]any{"type": "dropdown", "sourceRange": map[string]any{"a1Notation": "T1:T3"}},
map[string]any{"type": "dropdown", "sourceRange": map[string]any{"sheetId": "source"}},
map[string]any{"type": "dropdown", "sourceRange": map[string]any{"sheetId": "source", "a1Notation": "T1:T3", "colors": []any{"#fff"}}},
map[string]any{"type": "dropdown", "sourceRange": map[string]any{"sheetId": "source", "a1Notation": "T1:T3"}, "enableMultiSelect": "yes"},
}
for _, value := range invalid {
if err := validateDataValidation(value, "dv"); err == nil {
t.Errorf("invalid SourceRange data validation %#v returned nil", value)
}
}
}
+18 -1
View File
@@ -26,6 +26,16 @@ func newRangeCmd() *cobra.Command {
value 单元格值(内容由 --value-render-option 决定)
dataValidation 数据验证配置(下拉列表/复选框),无则为 null
hyperlink 单元格级超链接(path/sheet/range),无则省略
顶层还会返回 rowIndices / colIndices 与完成度字段:
hasMore 目标范围是否还有未返回数据
truncationReasons 部分返回原因(max_cells)
resolvedRange 未传 --range 时底层解析出的完整目标范围
returnedRange 本次实际完整返回的范围
单次最多返回 30,000 个单元格。hasMore=true 是部分成功,本命令
不会自动续读;请结合目标范围和 returnedRange 从下一行显式传
--range。收到 forbidden.document.sizeOverLimit 则表示工作簿整体无法装载,
应创建更小副本或拆分工作簿,缩小 --range 不能解决。
注意:range read/get 不返回合并单元格结构。查看合并范围请使用
dws sheet info --node NODE_ID --sheet-id SHEET_ID --format json,并读取 mergedRanges。
@@ -157,7 +167,12 @@ dws sheet info --node NODE_ID --sheet-id SHEET_ID --format json,并读取 merg
2) {"dataValidation":{"type":"none"}} → 显式清除该单元格 DV
3) {"dataValidation":{"type":"dropdown",...}} → 写新 dropdown(覆盖)
{"dataValidation":{"type":"checkbox",...}} → 写新 checkbox(覆盖)
dropdown: {"dataValidation":{"type":"dropdown","options":[{"value":"选项1"}],"enableMultiSelect":false}}
Inline dropdown:
{"dataValidation":{"type":"dropdown","options":[{"value":"选项1"}],"enableMultiSelect":false}}
SourceRange dropdown(同一工作簿内可跨工作表;不展开来源 values,不支持 colors):
{"dataValidation":{"type":"dropdown","sourceRange":{"sheetId":"SOURCE_SHEET_ID","a1Notation":"T1:T3"},"enableMultiSelect":false}}
dropdown 的 options 与 sourceRange 必须且只能传一个。SourceRange 支持普通区域、整行和整列;
a1Notation 不带工作表前缀,来源工作表单独写在 sheetId。
checkbox: {"dataValidation":{"type":"checkbox","checked":true}}
可与 text/richText 共存,也可单独使用(如 {dataValidation:{type:"none"}} 仅清除 DV 不写值)
@@ -169,6 +184,8 @@ dws sheet info --node NODE_ID --sheet-id SHEET_ID --format json,并读取 merg
- 只设样式或批量刷整片区域样式请用 dws sheet range set-style;写值同时设置少量 cell 样式可用 cellStyles
- 目标范围与已有合并区域冲突时,range update 会返回 MERGED_CELLS_CONFLICT;先用 sheet info 查看 mergedRanges,取消合并后写入,必要时再重新合并
- csv-put 的合并处理不同:目标区域含合并单元格时会打散合并并写入 CSV 值或公式
- SourceRange 在已验证的重命名、引用前插入行/列、删除引用前行的场景会自动调整;move-dimension 及其他未覆盖的删除/移动场景后先回读 sourceRangeStatus,仅 invalid 时重新选源写入
- 同一 cell 的 value/style 已写入但 SourceRange 校验失败时,服务端可返回 success=true,并通过 message 明确下拉未创建;必须检查 message,必要时重新读取确认
- 清空整片区域请用 dws sheet range clear`,
Example: ` # 写入文本
dws sheet range update --node NODE_ID --sheet-id SHEET_ID --range "A1:B2" \
@@ -0,0 +1,202 @@
package helpers
import (
"bytes"
"encoding/json"
"errors"
"strings"
"testing"
)
func executeSheetReadCompletionCommand(t *testing.T, caller *scriptedToolCaller, args ...string) (*bytes.Buffer, error) {
t.Helper()
installScriptedCaller(t, caller)
installSheetProductArgs(t)
stdout := &bytes.Buffer{}
deps.Out.w = stdout
cmd := newSheetCommand()
cmd.SilenceErrors = true
cmd.SilenceUsage = true
cmd.SetArgs(args)
return stdout, cmd.Execute()
}
func decodeSheetReadOutput(t *testing.T, stdout *bytes.Buffer) map[string]any {
t.Helper()
var got map[string]any
if err := json.Unmarshal(stdout.Bytes(), &got); err != nil {
t.Fatalf("decode sheet read output %q: %v", stdout.String(), err)
}
return got
}
func assertSheetReadCompletionMetadata(t *testing.T, got map[string]any) {
t.Helper()
if hasMore, ok := got["hasMore"].(bool); !ok || !hasMore {
t.Fatalf("hasMore = %#v, want true", got["hasMore"])
}
if got["resolvedRange"] != "A1:A30001" {
t.Fatalf("resolvedRange = %#v", got["resolvedRange"])
}
if got["returnedRange"] != "A1:A30000" {
t.Fatalf("returnedRange = %#v", got["returnedRange"])
}
reasons, ok := got["truncationReasons"].([]any)
if !ok || len(reasons) == 0 || reasons[0] != "max_cells" {
t.Fatalf("truncationReasons = %#v", got["truncationReasons"])
}
}
// csv-get intentionally projects the MCP response rather than inventing a
// client-side page model. Keep all completion metadata visible and make one
// request only, even when the server reports a partial success.
func TestSheetCSVGetPreservesCompletionMetadataWithoutAutoPagination(t *testing.T) {
const response = `{"csv":"[row=1]1\n","colIndices":["A"],"rowIndices":[1],"hasMore":true,"truncationReasons":["max_cells","max_chars"],"resolvedRange":"A1:A30001","returnedRange":"A1:A30000","message":"partial"}`
caller := &scriptedToolCaller{
format: "raw",
steps: []scriptedToolStep{{text: response}},
}
stdout, err := executeSheetReadCompletionCommand(t, caller,
"csv-get", "--node", "NODE")
if err != nil {
t.Fatalf("csv-get: %v", err)
}
if caller.calls != 1 || caller.tool != "get_range_as_csv" {
t.Fatalf("calls/tool = %d/%q, want one get_range_as_csv call", caller.calls, caller.tool)
}
if _, exists := caller.args["range"]; exists {
t.Fatalf("omitted range was unexpectedly synthesized by the CLI: %#v", caller.args["range"])
}
if got := strings.TrimSpace(stdout.String()); got != response {
t.Fatalf("raw csv-get response changed:\n got: %s\nwant: %s", got, response)
}
got := decodeSheetReadOutput(t, stdout)
assertSheetReadCompletionMetadata(t, got)
reasons := got["truncationReasons"].([]any)
if len(reasons) != 2 || reasons[1] != "max_chars" {
t.Fatalf("truncationReasons = %#v, want max_cells and max_chars", reasons)
}
}
// range read parses the get_cell_infos envelope to remove empty per-cell
// schema shells. That cleanup must not discard new top-level completion fields.
func TestSheetRangeReadPreservesCompletionMetadataWithoutAutoPagination(t *testing.T) {
const response = `{"cells":[[{"value":"1","dataValidation":{"type":null},"hyperlink":{"url":null}}]],"colIndices":["A"],"rowIndices":[1],"hasMore":true,"truncationReasons":["max_cells"],"resolvedRange":"A1:A30001","returnedRange":"A1:A30000","message":"partial"}`
caller := &scriptedToolCaller{
format: "json",
steps: []scriptedToolStep{{text: response}},
}
stdout, err := executeSheetReadCompletionCommand(t, caller,
"range", "read", "--node", "NODE")
if err != nil {
t.Fatalf("range read: %v", err)
}
if caller.calls != 1 || caller.tool != "get_cell_infos" {
t.Fatalf("calls/tool = %d/%q, want one get_cell_infos call", caller.calls, caller.tool)
}
if _, exists := caller.args["range"]; exists {
t.Fatalf("omitted range was unexpectedly synthesized by the CLI: %#v", caller.args["range"])
}
got := decodeSheetReadOutput(t, stdout)
assertSheetReadCompletionMetadata(t, got)
cells := got["cells"].([]any)
row := cells[0].([]any)
cell := row[0].(map[string]any)
if _, exists := cell["dataValidation"]; exists {
t.Fatalf("empty dataValidation shell was not removed: %#v", cell)
}
if _, exists := cell["hyperlink"]; exists {
t.Fatalf("empty hyperlink shell was not removed: %#v", cell)
}
}
// sheet +read calls get_cell_infos through this direct passthrough instead of
// callMCPToolCellInfos. Lock that third read surface to the same raw envelope
// contract and to one request only.
func TestSheetCellInfosDirectPassthroughPreservesCompletionMetadata(t *testing.T) {
const response = `{"cells":[[{"value":"1"}]],"colIndices":["A"],"rowIndices":[1],"hasMore":true,"truncationReasons":["max_cells"],"resolvedRange":"A1:A30001","returnedRange":"A1:A30000","message":"partial"}`
caller := &scriptedToolCaller{
format: "raw",
steps: []scriptedToolStep{{text: response}},
}
installScriptedCaller(t, caller)
stdout := &bytes.Buffer{}
deps.Out.w = stdout
if err := CallMCPToolOnServer("sheet", "get_cell_infos", map[string]any{"nodeId": "NODE"}); err != nil {
t.Fatalf("direct get_cell_infos: %v", err)
}
if caller.calls != 1 || caller.server != "sheet" || caller.tool != "get_cell_infos" {
t.Fatalf("calls/server/tool = %d/%q/%q", caller.calls, caller.server, caller.tool)
}
if got := strings.TrimSpace(stdout.String()); got != response {
t.Fatalf("raw get_cell_infos response changed:\n got: %s\nwant: %s", got, response)
}
assertSheetReadCompletionMetadata(t, decodeSheetReadOutput(t, stdout))
}
// Large-CP failure is not a pageable success. Both the raw csv-get path and
// both get_cell_infos paths must retain the backend code and safe user message
// exactly, matching the generic get_all_sheets error-classification path.
func TestSheetReadPreservesWorkbookSizeOverLimitError(t *testing.T) {
const response = `{"success":false,"errorCode":"forbidden.document.sizeOverLimit","errorMessage":"The workbook data is too large to process. Use a smaller copy or split the workbook, then try again."}`
tests := []struct {
name string
args []string
tool string
direct bool
}{
{name: "csv-get", args: []string{"csv-get", "--node", "NODE"}, tool: "get_range_as_csv"},
{name: "range-read", args: []string{"range", "read", "--node", "NODE"}, tool: "get_cell_infos"},
{name: "cell-infos-direct", tool: "get_cell_infos", direct: true},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
caller := &scriptedToolCaller{
format: "json",
steps: []scriptedToolStep{{text: response}},
}
var stdout *bytes.Buffer
var err error
if test.direct {
installScriptedCaller(t, caller)
stdout = &bytes.Buffer{}
deps.Out.w = stdout
err = CallMCPToolOnServer("sheet", test.tool, map[string]any{"nodeId": "NODE"})
} else {
stdout, err = executeSheetReadCompletionCommand(t, caller, test.args...)
}
if err == nil {
t.Fatal("large-CP response was accepted as success")
}
var cliErr *CLIError
if !errors.As(err, &cliErr) || cliErr.Code != CodeMCPToolError {
t.Fatalf("error = %T %v, want MCP tool error", err, err)
}
if cliErr.Message != response {
t.Fatalf("backend error payload changed:\n got: %s\nwant: %s", cliErr.Message, response)
}
for _, want := range []string{
"forbidden.document.sizeOverLimit",
"The workbook data is too large to process.",
} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not preserve %q", err, want)
}
}
if stdout.Len() != 0 {
t.Fatalf("failure wrote a success body: %q", stdout.String())
}
if caller.calls != 1 || caller.tool != test.tool {
t.Fatalf("calls/tool = %d/%q, want one %s call", caller.calls, caller.tool, test.tool)
}
})
}
}
+40 -14
View File
@@ -239,22 +239,48 @@ func validateDataValidation(dvRaw any, path string) error {
}
switch dvType {
case "dropdown":
optionsRaw, ok := dv["options"]
if !ok {
return fmt.Errorf("%s: type=dropdown 必须包含 options 数组", path)
optionsRaw, hasOptions := dv["options"]
sourceRangeRaw, hasSourceRange := dv["sourceRange"]
hasOptions = hasOptions && optionsRaw != nil
hasSourceRange = hasSourceRange && sourceRangeRaw != nil
if hasOptions == hasSourceRange {
return fmt.Errorf("%s: type=dropdown 的 options 与 sourceRange 必须且只能包含一个", path)
}
options, ok := optionsRaw.([]any)
if !ok || len(options) == 0 {
return fmt.Errorf("%s.options 必须为非空数组", path)
}
for i, opt := range options {
optMap, ok := opt.(map[string]any)
if !ok {
return fmt.Errorf("%s.options[%d] 必须为 object(如 {\"value\":\"选项\"})", path, i)
if hasOptions {
options, ok := optionsRaw.([]any)
if !ok || len(options) == 0 {
return fmt.Errorf("%s.options 必须为非空数组", path)
}
val, ok := optMap["value"].(string)
if !ok || val == "" {
return fmt.Errorf("%s.options[%d].value 必须为非空字符串", path, i)
for i, opt := range options {
optMap, ok := opt.(map[string]any)
if !ok {
return fmt.Errorf("%s.options[%d] 必须为 object(如 {\"value\":\"选项\"})", path, i)
}
val, ok := optMap["value"].(string)
if !ok || val == "" {
return fmt.Errorf("%s.options[%d].value 必须为非空字符串", path, i)
}
}
} else {
sourceRange, ok := sourceRangeRaw.(map[string]any)
if !ok {
return fmt.Errorf("%s.sourceRange 必须为 object", path)
}
if _, exists := sourceRange["colors"]; exists {
return fmt.Errorf("%s.sourceRange.colors 暂不支持", path)
}
sheetID, ok := sourceRange["sheetId"].(string)
if !ok || strings.TrimSpace(sheetID) == "" {
return fmt.Errorf("%s.sourceRange.sheetId 必须为非空字符串", path)
}
a1Notation, ok := sourceRange["a1Notation"].(string)
if !ok || strings.TrimSpace(a1Notation) == "" {
return fmt.Errorf("%s.sourceRange.a1Notation 必须为非空字符串", path)
}
}
if multi, exists := dv["enableMultiSelect"]; exists {
if _, ok := multi.(bool); !ok {
return fmt.Errorf("%s.enableMultiSelect 必须为 boolean", path)
}
}
case "checkbox":
+1 -1
View File
@@ -733,7 +733,7 @@ var RecordQuery = shortcut.Shortcut{
if rt.Changed("cursor") {
params["cursor"] = rt.Str("cursor")
}
return rt.CallMCP("query_records", params)
return executeRecordQuery(rt, params)
},
}
+24 -11
View File
@@ -98,7 +98,7 @@ func executeRecordDeleteBatches(rt *shortcut.RuntimeContext) error {
step.Status = "unknown"
step.Error = writeErr.Error()
}
remaining, verifyErr := queryRecordsByIDs(rt, baseID, tableID, batch)
remaining, verifyErr := queryDeletedRecordsByIDs(rt, baseID, tableID, batch)
if verifyErr == nil && len(remaining) > 0 {
verifyErr = fmt.Errorf("read-back still contains deleted record IDs: %s", strings.Join(recordIDs(remaining), ","))
}
@@ -303,20 +303,33 @@ func verifyUpsertBatch(rt *shortcut.RuntimeContext, baseID, tableID string, batc
}
func queryRecordsByIDs(rt *shortcut.RuntimeContext, baseID, tableID string, ids []string) ([]map[string]any, error) {
data, err := rt.CallMCPData(serverMain, "query_records", map[string]any{
"baseId": baseID, "tableId": tableID, "recordIds": ids, "limit": len(ids),
})
window, err := queryRecordWindow(rt, map[string]any{
"baseId": baseID, "tableId": tableID, "recordIds": ids,
}, len(ids))
if err != nil {
return nil, err
}
records, found := findRecords(data)
if !found {
return nil, fmt.Errorf("query_records read-back is missing records")
// Exact-ID verification below compares every requested ID with the returned
// records. The service can publish a continuation even after all requested
// IDs are present, so hasMore is not evidence that this bounded read-back is
// incomplete.
return window.Records, nil
}
func queryDeletedRecordsByIDs(rt *shortcut.RuntimeContext, baseID, tableID string, ids []string) ([]map[string]any, error) {
remaining := make([]map[string]any, 0)
for offset := 0; offset < len(ids); offset += recordQueryServicePageSize {
end := minInt(offset+recordQueryServicePageSize, len(ids))
chunk := ids[offset:end]
window, err := queryRecordWindow(rt, map[string]any{
"baseId": baseID, "tableId": tableID, "recordIds": chunk,
}, len(chunk))
if err != nil {
return nil, err
}
remaining = append(remaining, window.Records...)
}
if responseHasMore(data) {
return nil, fmt.Errorf("query_records read-back is incomplete")
}
return records, nil
return remaining, nil
}
func recordIDs(records []map[string]any) []string {
+233 -1
View File
@@ -10,13 +10,245 @@ import (
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/shortcut"
)
// query_records is backed by a service pager whose stable typed-envelope
// boundary is 20 records. Asking that layer to aggregate more than one page can
// drop the envelope fields used by the record projector, turning a successful
// read into an empty collection. Keep every remote request on one service page
// and aggregate only after each page has passed the CLI's shape validation.
const (
recordQueryServicePageSize = 20
recordQueryMaxConsecutiveEmptyPages = 3
)
type recordQueryWindow struct {
Records []map[string]any
HasMore bool
NextCursor string
Pages int
TotalCount any
HasTotalCount bool
}
func queryRecordWindow(rt *shortcut.RuntimeContext, params map[string]any, limit int) (recordQueryWindow, error) {
if limit <= 0 {
return recordQueryWindow{}, fmt.Errorf("record query limit must be positive, got %d", limit)
}
request := cloneAnyMap(params)
cursor, _ := request["cursor"].(string)
cursor = strings.TrimSpace(cursor)
seen := map[string]bool{}
if cursor != "" {
seen[cursor] = true
}
window := recordQueryWindow{Records: make([]map[string]any, 0, limit)}
consecutiveEmptyPages := 0
for len(window.Records) < limit {
pageSize := minInt(recordQueryServicePageSize, limit-len(window.Records))
request["limit"] = pageSize
if cursor == "" {
delete(request, "cursor")
} else {
request["cursor"] = cursor
}
data, err := rt.CallMCPData(serverMain, "query_records", request)
if err != nil {
return recordQueryWindow{}, err
}
if !window.HasTotalCount {
window.TotalCount, window.HasTotalCount = responseTotalCount(data)
}
records, found := findRecords(data)
if !found {
if explicitEmptyRecordQuery(data) {
window.Pages++
window.HasMore = false
window.NextCursor = ""
return window, nil
}
return recordQueryWindow{}, fmt.Errorf("query_records page %d is missing the records collection", window.Pages+1)
}
window.Records = append(window.Records, records...)
window.Pages++
if !responseHasMore(data) {
window.HasMore = false
window.NextCursor = ""
return window, nil
}
if len(records) == 0 {
consecutiveEmptyPages++
if consecutiveEmptyPages >= recordQueryMaxConsecutiveEmptyPages {
return recordQueryWindow{}, fmt.Errorf(
"query_records made no progress for %d consecutive pages",
consecutiveEmptyPages,
)
}
} else {
consecutiveEmptyPages = 0
}
next := responseCursor(data)
if next == "" {
return recordQueryWindow{}, fmt.Errorf("query_records page %d reports more data but no next cursor", window.Pages)
}
if seen[next] {
return recordQueryWindow{}, fmt.Errorf("query_records cursor cycle detected at %q", next)
}
seen[next] = true
cursor = next
window.HasMore = true
window.NextCursor = next
}
return window, nil
}
// explicitEmptyRecordQuery recognizes the service's reviewed zero-match wire
// shape. It is deliberately stricter than "records is absent": only a complete
// success envelope with an empty data object is accepted as an empty set.
func explicitEmptyRecordQuery(data map[string]any) bool {
if data == nil || data["success"] != true || data["status"] != "success" {
return false
}
payload, ok := data["data"].(map[string]any)
if !ok || len(payload) != 0 {
return false
}
if rawError, exists := data["error"]; exists && rawError != nil {
errorObject, ok := rawError.(map[string]any)
if !ok || len(errorObject) != 0 {
return false
}
}
return true
}
func executeRecordQuery(rt *shortcut.RuntimeContext, params map[string]any) error {
limit := 100
if rt.Changed("limit") {
limit = rt.Int("limit")
}
if limit < 1 || limit > recordBatchSize {
return fmt.Errorf("--limit must be in [1,%d], got %d", recordBatchSize, limit)
}
requestedIDs, exactIDQuery, err := recordQueryRequestedIDs(params)
if err != nil {
return err
}
if exactIDQuery {
if len(requestedIDs) > recordBatchSize {
return fmt.Errorf("--record-ids accepts at most %d unique IDs, got %d", recordBatchSize, len(requestedIDs))
}
params = cloneAnyMap(params)
params["recordIds"] = requestedIDs
limit = minInt(limit, len(requestedIDs))
}
if rt.DryRun() {
return rt.Output(map[string]any{
"dry_run": true,
"executed": false,
"tool": "query_records",
"arguments": params,
})
}
window, err := queryRecordWindow(rt, params, limit)
if err != nil {
return err
}
if exactIDQuery {
complete, err := validateExactRecordQuery(window.Records, requestedIDs)
if err != nil {
return err
}
if complete {
// The service may advertise unrelated continuation after every
// requested ID is already present. Exact-ID completion is stronger
// evidence than that residual pager state.
window.HasMore = false
window.NextCursor = ""
}
}
records := make([]any, 0, len(window.Records))
for _, record := range window.Records {
records = append(records, record)
}
data := map[string]any{
"records": records,
"hasMore": window.HasMore,
"page": window.Pages,
"size": len(window.Records),
}
if window.NextCursor != "" {
data["nextCursor"] = window.NextCursor
}
if window.HasTotalCount {
data["totalCount"] = window.TotalCount
}
return rt.Output(map[string]any{
"success": true,
"status": "success",
"data": data,
})
}
func responseTotalCount(data map[string]any) (any, bool) {
for {
if totalCount, exists := data["totalCount"]; exists {
return totalCount, true
}
nested, ok := data["data"].(map[string]any)
if !ok {
return nil, false
}
data = nested
}
}
func recordQueryRequestedIDs(params map[string]any) ([]string, bool, error) {
raw, exists := params["recordIds"]
if !exists {
return nil, false, nil
}
values, ok := raw.([]string)
if !ok {
return nil, false, fmt.Errorf("recordIds must be a string list, got %T", raw)
}
ids, err := parseRecordIDs(values)
if err != nil {
return nil, false, err
}
return ids, true, nil
}
func validateExactRecordQuery(records []map[string]any, requestedIDs []string) (bool, error) {
wanted := make(map[string]bool, len(requestedIDs))
for _, id := range requestedIDs {
wanted[id] = true
}
seen := make(map[string]bool, len(records))
for index, record := range records {
id := recordID(record)
if id == "" {
return false, fmt.Errorf("query_records exact-ID result %d is missing recordId", index)
}
if !wanted[id] {
return false, fmt.Errorf("query_records exact-ID result contains unexpected recordId %q", id)
}
if seen[id] {
return false, fmt.Errorf("query_records exact-ID result contains duplicate recordId %q", id)
}
seen[id] = true
}
return len(seen) == len(wanted), nil
}
func queryAllRecords(rt *shortcut.RuntimeContext, params map[string]any, maxRecords int) ([]map[string]any, error) {
all := make([]map[string]any, 0)
cursor := ""
seen := map[string]bool{}
for page := 0; ; page++ {
request := cloneAnyMap(params)
request["limit"] = recordBatchSize
request["limit"] = recordQueryServicePageSize
if cursor != "" {
request["cursor"] = cursor
}
@@ -9,6 +9,7 @@ import (
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
"testing"
@@ -265,6 +266,446 @@ func recordListJSON(t *testing.T, records []map[string]any) string {
return string(raw)
}
func pagedRecordQueryResponse(t *testing.T, records []map[string]any, args map[string]any) string {
t.Helper()
limit, ok := args["limit"].(int)
if !ok || limit < 1 || limit > recordQueryServicePageSize {
t.Fatalf("query_records limit = %#v, want 1..%d", args["limit"], recordQueryServicePageSize)
}
offset := 0
if cursor, _ := args["cursor"].(string); cursor != "" {
if _, err := fmt.Sscanf(cursor, "offset-%d", &offset); err != nil {
t.Fatalf("invalid test cursor %q: %v", cursor, err)
}
}
end := minInt(offset+limit, len(records))
pageRecords := records[offset:end]
data := map[string]any{"records": pageRecords, "hasMore": end < len(records), "totalCount": len(records)}
if end < len(records) {
data["nextCursor"] = fmt.Sprintf("offset-%d", end)
}
raw, err := json.Marshal(map[string]any{
"success": true,
"hasMore": end < len(records),
"page": offset/recordQueryServicePageSize + 1,
"size": limit,
"data": data,
})
if err != nil {
t.Fatal(err)
}
return string(raw)
}
func runRecordQueryShortcutCLI(t *testing.T, caller *upsertByKeyCaller, limit int, extra ...string) (map[string]any, error) {
t.Helper()
helpers.InitDepsForTest(t, caller)
root := &cobra.Command{Use: "dws", SilenceErrors: true, SilenceUsage: true}
root.PersistentFlags().Bool("yes", false, "")
root.PersistentFlags().Bool("dry-run", false, "")
root.PersistentFlags().String("format", "json", "")
root.AddCommand(shortcut.Commands()...)
stdout := &bytes.Buffer{}
root.SetOut(stdout)
root.SetErr(&bytes.Buffer{})
args := []string{"aitable", "+record-query", "--base-id", "base", "--table-id", "table", "--limit", fmt.Sprint(limit)}
root.SetArgs(append(args, extra...))
err := root.Execute()
if stdout.Len() == 0 {
return nil, err
}
var payload map[string]any
if decodeErr := json.Unmarshal(stdout.Bytes(), &payload); decodeErr != nil {
t.Fatalf("decode record query output %q: %v", stdout.String(), decodeErr)
}
return payload, err
}
func TestCrossPlatformCoverageRecordQueryServicePageBoundariesE2E(t *testing.T) {
for _, size := range []int{20, 21, 22, 100} {
t.Run(fmt.Sprintf("size_%d", size), func(t *testing.T) {
records := updateFixtureRecords(0, size, "可见")
caller := &upsertByKeyCaller{}
caller.callFn = func(_ int, _, tool string, args map[string]any) (string, error) {
if tool != "query_records" {
return "", fmt.Errorf("unexpected tool %s", tool)
}
return pagedRecordQueryResponse(t, records, args), nil
}
payload, err := runRecordQueryShortcutCLI(t, caller, size)
if err != nil {
t.Fatalf("record query size %d: %v", size, err)
}
data, ok := payload["data"].(map[string]any)
if !ok || payload["success"] != true || data["hasMore"] != false || data["totalCount"] != float64(size) || len(data["records"].([]any)) != size {
t.Fatalf("record query size %d payload = %#v", size, payload)
}
wantCalls := (size + recordQueryServicePageSize - 1) / recordQueryServicePageSize
if len(caller.calls) != wantCalls || data["page"] != float64(wantCalls) || data["size"] != float64(size) {
t.Fatalf("record query size %d calls=%d payload=%#v, want calls=%d", size, len(caller.calls), payload, wantCalls)
}
})
}
}
func TestCrossPlatformCoverageRecordQueryDryRunStopsBeforeTransportE2E(t *testing.T) {
caller := &upsertByKeyCaller{}
payload, err := runRecordQueryShortcutCLI(t, caller, 5, "--query", "needle", "--dry-run")
if err != nil {
t.Fatalf("record query dry-run error = %v", err)
}
if len(caller.calls) != 0 {
t.Fatalf("record query dry-run crossed transport: %#v", caller.calls)
}
arguments, _ := payload["arguments"].(map[string]any)
if payload["dry_run"] != true || payload["executed"] != false || payload["tool"] != "query_records" ||
arguments["baseId"] != "base" || arguments["tableId"] != "table" || arguments["keyword"] != "needle" || arguments["limit"] != float64(5) {
t.Fatalf("record query dry-run payload = %#v", payload)
}
}
func TestCrossPlatformCoverageRecordQueryDryRunRejectsInvalidLocalPlanE2E(t *testing.T) {
for _, testCase := range []struct {
name string
limit int
extra []string
want string
}{
{name: "zero limit", limit: 0, extra: []string{"--dry-run"}, want: "--limit must be in [1,100]"},
{name: "excessive limit", limit: recordBatchSize + 1, extra: []string{"--dry-run"}, want: "--limit must be in [1,100]"},
{name: "empty record IDs", limit: 100, extra: []string{"--record-ids", " ", "--dry-run"}, want: "至少包含一个非空 recordId"},
{
name: "excessive record IDs",
limit: 100,
extra: []string{"--record-ids", strings.Join(recordIDFixtures(recordBatchSize+1), ","), "--dry-run"},
want: "at most 100 unique IDs",
},
} {
t.Run(testCase.name, func(t *testing.T) {
caller := &upsertByKeyCaller{}
payload, err := runRecordQueryShortcutCLI(t, caller, testCase.limit, testCase.extra...)
if err == nil || !strings.Contains(err.Error(), testCase.want) {
t.Fatalf("record query dry-run error = %v, want %q (payload=%#v)", err, testCase.want, payload)
}
if len(caller.calls) != 0 {
t.Fatalf("invalid record query dry-run crossed transport: %#v", caller.calls)
}
})
}
}
func TestCrossPlatformCoverageRecordQueryExactIDsIgnoreResidualContinuationE2E(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(call int, _, tool string, args map[string]any) (string, error) {
if tool != "query_records" {
return "", fmt.Errorf("unexpected tool %s", tool)
}
if call != 0 {
return "", errors.New("exact-ID query followed residual continuation")
}
if args["limit"] != 2 {
t.Fatalf("exact-ID service limit = %#v, want 2", args["limit"])
}
ids, ok := args["recordIds"].([]string)
if !ok || !slices.Equal(ids, []string{"r1", "r2"}) {
t.Fatalf("exact-ID request IDs = %#v", args["recordIds"])
}
return `{"success":true,"data":{"records":[{"recordId":"r1"},{"recordId":"r2"}],"hasMore":true,"nextCursor":"residual"}}`, nil
}}
payload, err := runRecordQueryShortcutCLI(t, caller, 100, "--record-ids", "r2,r1,r2")
data, _ := payload["data"].(map[string]any)
if err != nil || len(caller.calls) != 1 || data["hasMore"] != false || data["size"] != float64(2) {
t.Fatalf("exact-ID payload=%#v calls=%d err=%v", payload, len(caller.calls), err)
}
if _, exists := data["nextCursor"]; exists {
t.Fatalf("exact-ID payload retained residual cursor: %#v", payload)
}
}
func TestCrossPlatformCoverageRecordQueryFailureAndContinuationBranches(t *testing.T) {
t.Run("non-positive window limit", func(t *testing.T) {
caller := &upsertByKeyCaller{}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
if _, err := queryRecordWindow(rt, nil, 0); err == nil {
t.Fatal("non-positive record query window limit accepted")
}
})
t.Run("initial cursor cycle", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(_ int, _, tool string, _ map[string]any) (string, error) {
if tool != "query_records" {
return "", fmt.Errorf("unexpected tool %s", tool)
}
return `{"success":true,"data":{"records":[{"recordId":"r1"}],"hasMore":true,"nextCursor":"seed"}}`, nil
}}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
if _, err := queryRecordWindow(rt, map[string]any{"cursor": " seed "}, 2); err == nil || !strings.Contains(err.Error(), "cursor cycle") {
t.Fatalf("initial cursor cycle error = %v", err)
}
})
t.Run("strict explicit empty shape", func(t *testing.T) {
for _, data := range []map[string]any{
{"success": true, "status": "success", "data": map[string]any{"unexpected": true}},
{"success": true, "status": "success", "data": map[string]any{}, "error": "bad"},
} {
if explicitEmptyRecordQuery(data) {
t.Fatalf("invalid empty-query envelope accepted: %#v", data)
}
}
})
t.Run("explicit empty page", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(_ int, _, _ string, _ map[string]any) (string, error) {
return `{"success":true,"status":"success","error":{},"data":{}}`, nil
}}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
window, err := queryRecordWindow(rt, map[string]any{}, 1)
if err != nil || len(window.Records) != 0 || window.Pages != 1 || window.HasMore || window.NextCursor != "" {
t.Fatalf("explicit empty window=%#v err=%v", window, err)
}
})
t.Run("missing records collection", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(_ int, _, _ string, _ map[string]any) (string, error) {
return `{}`, nil
}}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
if _, err := queryRecordWindow(rt, map[string]any{}, 1); err == nil || !strings.Contains(err.Error(), "missing the records collection") {
t.Fatalf("missing records error = %v", err)
}
})
t.Run("changing cursors cannot hide sustained empty pages", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(call int, _, tool string, _ map[string]any) (string, error) {
if tool != "query_records" {
return "", fmt.Errorf("unexpected tool %s", tool)
}
return fmt.Sprintf(
`{"success":true,"data":{"records":[],"hasMore":true,"nextCursor":"empty-%d"}}`,
call,
), nil
}}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
if _, err := queryRecordWindow(rt, map[string]any{}, 1); err == nil || !strings.Contains(err.Error(), "no progress") {
t.Fatalf("sustained empty pages error = %v", err)
}
if len(caller.calls) != recordQueryMaxConsecutiveEmptyPages {
t.Fatalf("query calls = %d, want %d", len(caller.calls), recordQueryMaxConsecutiveEmptyPages)
}
})
t.Run("invalid shortcut limit", func(t *testing.T) {
caller := &upsertByKeyCaller{}
helpers.InitDepsForTest(t, caller)
cmd := &cobra.Command{Use: "query"}
cmd.Flags().Int("limit", 0, "")
if err := cmd.Flags().Set("limit", "0"); err != nil {
t.Fatal(err)
}
rt := shortcut.RuntimeContextForTest(cmd, RecordQuery)
if err := executeRecordQuery(rt, map[string]any{}); err == nil {
t.Fatal("invalid shortcut limit accepted")
}
})
t.Run("invalid exact-ID request", func(t *testing.T) {
for _, testCase := range []struct {
name string
params map[string]any
want string
}{
{name: "wrong type", params: map[string]any{"recordIds": 1}, want: "string list"},
{name: "empty", params: map[string]any{"recordIds": []string{" "}}, want: "至少包含一个非空 recordId"},
{name: "over service limit", params: map[string]any{"recordIds": recordIDFixtures(recordBatchSize + 1)}, want: "at most 100 unique IDs"},
} {
t.Run(testCase.name, func(t *testing.T) {
caller := &upsertByKeyCaller{}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
err := executeRecordQuery(rt, testCase.params)
if err == nil || !strings.Contains(err.Error(), testCase.want) {
t.Fatalf("exact-ID request error = %v, want %q", err, testCase.want)
}
if len(caller.calls) != 0 {
t.Fatalf("invalid exact-ID request made %d service calls", len(caller.calls))
}
})
}
})
t.Run("invalid exact-ID response", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(_ int, _, tool string, _ map[string]any) (string, error) {
if tool != "query_records" {
return "", fmt.Errorf("unexpected tool %s", tool)
}
return `{"success":true,"data":{"records":[{"recordId":"other"}]}}`, nil
}}
if _, err := runRecordQueryShortcutCLI(t, caller, 100, "--record-ids", "r1"); err == nil || !strings.Contains(err.Error(), "unexpected recordId") {
t.Fatalf("unexpected exact-ID response error = %v", err)
}
if _, err := validateExactRecordQuery([]map[string]any{{}}, []string{"r1"}); err == nil || !strings.Contains(err.Error(), "missing recordId") {
t.Fatalf("missing exact-ID response error = %v", err)
}
duplicate := []map[string]any{{"recordId": "r1"}, {"recordId": "r1"}}
if _, err := validateExactRecordQuery(duplicate, []string{"r1", "r2"}); err == nil || !strings.Contains(err.Error(), "duplicate recordId") {
t.Fatalf("duplicate exact-ID response error = %v", err)
}
})
t.Run("shortcut query failure", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(_ int, _, _ string, _ map[string]any) (string, error) {
return "", errors.New("query unavailable")
}}
helpers.InitDepsForTest(t, caller)
rt := shortcut.RuntimeContextForTest(&cobra.Command{Use: "query"}, RecordQuery)
if err := executeRecordQuery(rt, map[string]any{}); err == nil || !strings.Contains(err.Error(), "query unavailable") {
t.Fatalf("shortcut query failure = %v", err)
}
})
t.Run("bounded continuation is published", func(t *testing.T) {
records := updateFixtureRecords(0, 21, "visible")
caller := &upsertByKeyCaller{callFn: func(_ int, _, tool string, args map[string]any) (string, error) {
if tool != "query_records" {
return "", fmt.Errorf("unexpected tool %s", tool)
}
return pagedRecordQueryResponse(t, records, args), nil
}}
payload, err := runRecordQueryShortcutCLI(t, caller, 20)
data, _ := payload["data"].(map[string]any)
if err != nil || data["hasMore"] != true || data["nextCursor"] != "offset-20" {
t.Fatalf("bounded continuation payload=%#v err=%v", payload, err)
}
})
t.Run("delete readback follows active continuation to absence", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(call int, _, tool string, args map[string]any) (string, error) {
if tool != "query_records" {
return `{"deletedCount":1}`, nil
}
if call == 1 {
if _, exists := args["cursor"]; exists {
t.Fatalf("first deletion readback unexpectedly has cursor: %#v", args)
}
return `{"success":true,"data":{"records":[],"hasMore":true,"nextCursor":"c1"}}`, nil
}
if args["cursor"] != "c1" {
t.Fatalf("continued deletion readback cursor = %#v", args["cursor"])
}
return `{"success":true,"data":{"records":[],"hasMore":false}}`, nil
}}
out, err := runRecordDeleteCLI(t, caller, []string{"r1"})
if err != nil || out == "" || len(caller.calls) != 3 {
t.Fatalf("delete continued absence output=%q error=%v calls=%#v", out, err, caller.calls)
}
})
t.Run("delete readback follows active continuation to remaining record", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(call int, _, tool string, _ map[string]any) (string, error) {
if tool != "query_records" {
return `{"deletedCount":1}`, nil
}
if call == 1 {
return `{"success":true,"data":{"records":[],"hasMore":true,"nextCursor":"c1"}}`, nil
}
return `{"success":true,"data":{"records":[{"recordId":"r1"}],"hasMore":false}}`, nil
}}
out, err := runRecordDeleteCLI(t, caller, []string{"r1"})
if err == nil || out != "" || len(caller.calls) != 3 {
t.Fatalf("delete continued remaining output=%q error=%v calls=%#v", out, err, caller.calls)
}
var typed *apperrors.Error
if !errors.As(err, &typed) || typed.Reason != "aitable_composite_unknown" {
t.Fatalf("delete continued remaining error = %#v", err)
}
})
t.Run("delete readback fails closed on empty continuation stall", func(t *testing.T) {
caller := &upsertByKeyCaller{callFn: func(call int, _, tool string, _ map[string]any) (string, error) {
if tool != "query_records" {
return `{"deletedCount":1}`, nil
}
return fmt.Sprintf(`{"success":true,"data":{"records":[],"hasMore":true,"nextCursor":"c%d"}}`, call), nil
}}
out, err := runRecordDeleteCLI(t, caller, []string{"r1"})
if err == nil || out != "" || len(caller.calls) != recordQueryMaxConsecutiveEmptyPages+1 {
t.Fatalf("delete stalled continuation output=%q error=%v calls=%#v", out, err, caller.calls)
}
var typed *apperrors.Error
if !errors.As(err, &typed) || typed.Reason != "aitable_composite_unknown" {
t.Fatalf("delete stalled continuation error = %#v", err)
}
})
}
func TestCrossPlatformCoverageRecordWriteReadbackUsesStableServicePagesE2E(t *testing.T) {
for _, operation := range []string{"update", "upsert", "delete"} {
for _, size := range []int{21, 22, 100} {
t.Run(fmt.Sprintf("%s_%d", operation, size), func(t *testing.T) {
records := updateFixtureRecords(0, size, "完成")
caller := &upsertByKeyCaller{}
queryCalls := 0
caller.callFn = func(_ int, _, tool string, args map[string]any) (string, error) {
if tool != "query_records" {
return `{"success":true}`, nil
}
queryCalls++
if operation == "delete" {
ids := args["recordIds"].([]string)
empty := make([]map[string]any, len(ids))
response := pagedRecordQueryResponse(t, empty, args)
var payload map[string]any
if err := json.Unmarshal([]byte(response), &payload); err != nil {
t.Fatal(err)
}
page := payload["data"].(map[string]any)
page["records"] = []any{}
raw, _ := json.Marshal(payload)
return string(raw), nil
}
response := pagedRecordQueryResponse(t, records, args)
var payload map[string]any
if err := json.Unmarshal([]byte(response), &payload); err != nil {
t.Fatal(err)
}
page := payload["data"].(map[string]any)
if len(page["records"].([]any))+recordQueryServicePageSize*(queryCalls-1) >= size {
// The live service can retain a continuation after it has
// returned every requested record ID. Exact-ID verification
// must use ID coverage, not this advisory bit.
page["hasMore"] = true
page["nextCursor"] = "ignored-after-complete-id-set"
payload["hasMore"] = true
}
raw, _ := json.Marshal(payload)
return string(raw), nil
}
var out string
var err error
if operation == "delete" {
out, err = runRecordDeleteCLI(t, caller, recordIDs(records))
} else {
out, err = runRecordBatchCLI(t, caller, "+record-"+operation, records)
}
if err != nil || out == "" {
t.Fatalf("%s size %d false negative: output=%q err=%v", operation, size, out, err)
}
wantQueries := (size + recordQueryServicePageSize - 1) / recordQueryServicePageSize
if queryCalls != wantQueries {
t.Fatalf("%s size %d query calls=%d, want %d", operation, size, queryCalls, wantQueries)
}
})
}
}
}
func TestCrossPlatformCoverageRecordUpdateAutoChunksAndVerifiesE2E(t *testing.T) {
records := updateFixtureRecords(0, 101, "完成")
caller := &upsertByKeyCaller{steps: []upsertByKeyStep{
@@ -395,6 +836,10 @@ func TestCrossPlatformCoverageRecordDeleteAutoChunksAndProvesAbsenceE2E(t *testi
caller := &upsertByKeyCaller{steps: []upsertByKeyStep{
{text: `{"deletedCount":100}`},
{text: `{"data":{"records":[]}}`},
{text: `{"data":{"records":[]}}`},
{text: `{"data":{"records":[]}}`},
{text: `{"data":{"records":[]}}`},
{text: `{"data":{"records":[]}}`},
{text: `{"deletedCount":1}`},
{text: `{"data":{"records":[]}}`},
}}
@@ -407,7 +852,7 @@ func TestCrossPlatformCoverageRecordDeleteAutoChunksAndProvesAbsenceE2E(t *testi
t.Fatalf("delete output missing %s: %s", want, out)
}
}
if len(caller.calls) != 4 || caller.calls[0].tool != "delete_records" || caller.calls[1].tool != "query_records" {
if len(caller.calls) != 8 || caller.calls[0].tool != "delete_records" || caller.calls[1].tool != "query_records" || caller.calls[6].tool != "delete_records" {
t.Fatalf("delete call sequence = %#v", caller.calls)
}
}
@@ -421,6 +866,17 @@ func TestCrossPlatformCoverageRecordDeleteEmptyReplyRecoveredOnlyByAbsenceE2E(t
}
})
t.Run("explicit service success with empty data proves deletion", func(t *testing.T) {
caller := &upsertByKeyCaller{steps: []upsertByKeyStep{
{text: `{"deletedCount":1}`},
{text: `{"success":true,"status":"success","error":{},"data":{}}`},
}}
out, err := runRecordDeleteCLI(t, caller, []string{"r1"})
if err != nil || !strings.Contains(out, `"status": "verified_absent"`) {
t.Fatalf("delete explicit empty success = output:%q err:%v", out, err)
}
})
t.Run("missing records contract is unknown", func(t *testing.T) {
caller := &upsertByKeyCaller{steps: []upsertByKeyStep{{text: `{"deletedCount":1}`}, {text: `{"data":{}}`}}}
out, err := runRecordDeleteCLI(t, caller, []string{"r1"})
@@ -450,6 +906,10 @@ func TestCrossPlatformCoverageRecordDeletePartialCheckpointE2E(t *testing.T) {
caller := &upsertByKeyCaller{steps: []upsertByKeyStep{
{text: `{"deletedCount":100}`},
{text: `{"records":[]}`},
{text: `{"records":[]}`},
{text: `{"records":[]}`},
{text: `{"records":[]}`},
{text: `{"records":[]}`},
{text: `{"deletedCount":0}`},
{text: `{"records":[{"recordId":"r100","cells":{}}]}`},
}}
+382 -33
View File
@@ -4,10 +4,14 @@
package doc
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"reflect"
"strings"
"time"
"unicode"
@@ -18,6 +22,9 @@ import (
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/localio"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/shortcut"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/shortcut/docresolver"
"github.com/yuin/goldmark"
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/renderer/html"
)
var (
@@ -28,6 +35,21 @@ var (
docMkdirTemp = os.MkdirTemp
docRemoveAll = os.RemoveAll
docDownload = localio.Download
docVerifyWait = waitForDocVerification
docVerifyDelays = []time.Duration{250 * time.Millisecond, 500 * time.Millisecond, time.Second, 2 * time.Second, 4 * time.Second, 8 * time.Second}
docMarkdown = goldmark.New(
goldmark.WithExtensions(extension.Table),
goldmark.WithRendererOptions(html.WithUnsafe()),
)
docMarkdownConvert = func(source []byte, writer io.Writer) error {
return docMarkdown.Convert(source, writer)
}
)
const (
docBlockReadPageSize = 50
docBlockReadMaxItems = 5000
docMarkdownVerifyMax = 2 * 1024 * 1024
)
var Create = shortcut.Shortcut{
@@ -133,7 +155,9 @@ var Create = shortcut.Shortcut{
verifyTool = "get_document_content"
verifyParams["format"] = format
}
verification, err := rt.CallMCPData(productDoc, verifyTool, verifyParams)
verification, err := readDocVerification(rt, verifyTool, verifyParams, func(data map[string]any) bool {
return content == "" || verifyUpdatedDocumentContent(data, content, "overwrite", format)
})
if err != nil {
return docVerificationError("doc.create", "verify", nodeID, err, append(steps, map[string]any{"name": "verify", "status": "failed"}))
}
@@ -422,7 +446,9 @@ var CheckpointUpdate = shortcut.Shortcut{
append(steps, map[string]any{"name": "update", "status": "failed"}, map[string]any{"name": "verify", "status": "not_started"}))
}
steps = append(steps, map[string]any{"name": "update", "status": "success"})
verification, err := rt.CallMCPData(productDoc, "get_document_content", map[string]any{"nodeId": rt.Str("node"), "format": "markdown"})
verification, err := readDocVerification(rt, "get_document_content", map[string]any{"nodeId": rt.Str("node"), "format": "markdown"}, func(data map[string]any) bool {
return verifyUpdatedDocumentContent(data, content, rt.Str("mode"), "markdown")
})
if err != nil {
return checkpointPartialWriteError(rt.Str("node"), checkpoint, "verify", "doc_checkpoint_verification_failed", err,
append(steps, map[string]any{"name": "verify", "status": "failed"}))
@@ -539,7 +565,7 @@ func executeUpdate(rt *shortcut.RuntimeContext) error {
}
referenceBlockID := rt.Str("after-block-id")
return executeVerifiedDocMutation(rt, "doc.update", "insert_document_block", params, node,
"list_document_blocks", map[string]any{"nodeId": node, "format": verificationFormat},
"list_document_blocks", map[string]any{"nodeId": node, "format": verificationFormat, "__allBlocks": true},
func(result, data map[string]any) bool {
return verifyInsertedBlock(result, data, referenceBlockID, content, rt.Str("doc-format"))
})
@@ -553,14 +579,14 @@ func executeUpdate(rt *shortcut.RuntimeContext) error {
params["element"] = map[string]any{"blockType": "paragraph", "paragraph": map[string]any{"text": content}}
}
return executeVerifiedDocMutation(rt, "doc.update", "update_document_block", params, node,
"list_document_blocks", map[string]any{"nodeId": node, "blockId": blockID, "format": verificationFormat},
"list_document_blocks", map[string]any{"nodeId": node, "format": verificationFormat, "__allBlocks": true},
func(_, data map[string]any) bool {
return blockContentEquals(data, blockID, content, rt.Str("doc-format"))
})
case "block_delete":
blockID := rt.Str("block-id")
return executeVerifiedDocMutation(rt, "doc.update", "delete_document_block", map[string]any{"nodeId": node, "blockId": blockID}, node,
"list_document_blocks", map[string]any{"nodeId": node, "format": "element"},
"list_document_blocks", map[string]any{"nodeId": node, "format": "element", "__allBlocks": true},
func(_, data map[string]any) bool { return findBlock(data, blockID) == nil })
case "str_replace":
return executePlainTextReplace(rt, node)
@@ -609,7 +635,7 @@ func nestedRevision(value any) (int, bool) {
}
func executePlainTextReplace(rt *shortcut.RuntimeContext, nodeID string) error {
data, err := rt.CallMCPData(productDoc, "list_document_blocks", map[string]any{"nodeId": nodeID, "format": "element"})
data, err := readAllDocumentBlocks(rt, map[string]any{"nodeId": nodeID, "format": "element"})
if err != nil {
return err
}
@@ -643,12 +669,12 @@ func executePlainTextReplace(rt *shortcut.RuntimeContext, nodeID string) error {
blockID := matches[0].blockID
return executeVerifiedDocMutation(rt, "doc.update", "update_document_block",
map[string]any{"nodeId": nodeID, "blockId": blockID, "element": map[string]any{"blockType": "paragraph", "paragraph": map[string]any{"text": updated}}}, nodeID,
"list_document_blocks", map[string]any{"nodeId": nodeID, "blockId": blockID, "format": "element"},
"list_document_blocks", map[string]any{"nodeId": nodeID, "format": "element", "__allBlocks": true},
func(_, data map[string]any) bool { return blockContentEquals(data, blockID, updated, "markdown") })
}
func executeBlockCopy(rt *shortcut.RuntimeContext, nodeID string) error {
data, err := rt.CallMCPData(productDoc, "list_document_blocks", map[string]any{"nodeId": nodeID, "blockId": rt.Str("block-id"), "format": "element"})
data, err := readAllDocumentBlocks(rt, map[string]any{"nodeId": nodeID, "format": "element"})
if err != nil {
return err
}
@@ -664,7 +690,7 @@ func executeBlockCopy(rt *shortcut.RuntimeContext, nodeID string) error {
referenceBlockID := rt.Str("after-block-id")
return executeVerifiedDocMutation(rt, "doc.update", "insert_document_block",
map[string]any{"nodeId": nodeID, "referenceBlockId": referenceBlockID, "where": "after", "element": block}, nodeID,
"list_document_blocks", map[string]any{"nodeId": nodeID, "format": "element"},
"list_document_blocks", map[string]any{"nodeId": nodeID, "format": "element", "__allBlocks": true},
func(result, data map[string]any) bool {
return verifyInsertedCanonicalBlock(result, data, referenceBlockID, expectedContent, "markdown")
})
@@ -684,7 +710,9 @@ func executeVerifiedDocMutation(
return docUnknownWriteError(operation, tool, nodeID, err)
}
steps[0]["status"] = "success"
verification, err := rt.CallMCPData(productDoc, verifyTool, verifyParams)
verification, err := readDocVerification(rt, verifyTool, verifyParams, func(data map[string]any) bool {
return verify == nil || verify(result, data)
})
if err != nil {
return docVerificationError(operation, "verify", nodeID, err, append(steps, map[string]any{"name": "verify", "status": "failed"}))
}
@@ -732,7 +760,9 @@ func executeVerifiedDocContentMutation(rt *shortcut.RuntimeContext, firstParams
}
steps = append(steps, map[string]any{"name": stepName, "status": "success"})
}
verification, err := rt.CallMCPData(productDoc, "get_document_content", map[string]any{"nodeId": nodeID, "format": format})
verification, err := readDocVerification(rt, "get_document_content", map[string]any{"nodeId": nodeID, "format": format}, func(data map[string]any) bool {
return verifyUpdatedDocumentContent(data, content, mode, format)
})
if err != nil {
return docVerificationError("doc.update", "verify", nodeID, err, append(steps, map[string]any{"name": "verify", "status": "failed"}))
}
@@ -745,6 +775,180 @@ func executeVerifiedDocContentMutation(rt *shortcut.RuntimeContext, firstParams
}, steps...))
}
func readDocVerification(rt *shortcut.RuntimeContext, tool string, rawParams map[string]any, verify func(map[string]any) bool) (map[string]any, error) {
params := cloneMap(rawParams)
allBlocks, _ := params["__allBlocks"].(bool)
delete(params, "__allBlocks")
var last map[string]any
var lastErr error
for attempt := 0; attempt <= len(docVerifyDelays); attempt++ {
var data map[string]any
var err error
if allBlocks && tool == "list_document_blocks" {
data, err = readAllDocumentBlocks(rt, params)
} else {
data, err = rt.CallMCPData(productDoc, tool, params)
}
if err != nil {
lastErr = err
} else {
last = data
lastErr = nil
if verify == nil || verify(data) {
return data, nil
}
}
if attempt < len(docVerifyDelays) {
if err := docVerifyWait(rt.Command().Context(), docVerifyDelays[attempt]); err != nil {
return nil, err
}
}
}
if lastErr != nil {
return nil, lastErr
}
return last, nil
}
func waitForDocVerification(ctx context.Context, delay time.Duration) error {
if ctx == nil {
ctx = context.Background()
}
timer := time.NewTimer(delay)
defer timer.Stop()
select {
case <-ctx.Done():
return ctx.Err()
case <-timer.C:
return nil
}
}
func readAllDocumentBlocks(rt *shortcut.RuntimeContext, base map[string]any) (map[string]any, error) {
all := make([]any, 0, docBlockReadPageSize)
seenPageIdentities := map[string]bool{}
for start := 0; start < docBlockReadMaxItems; start += docBlockReadPageSize {
params := cloneMap(base)
params["startIndex"] = start
params["endIndex"] = start + docBlockReadPageSize - 1
page, err := rt.CallMCPData(productDoc, "list_document_blocks", params)
if err != nil {
return nil, err
}
blocks, ok := documentBlockEntries(page)
if !ok {
return nil, fmt.Errorf("list_document_blocks 回读缺少 blocks 数组")
}
pageIdentity := documentBlockPageIdentity(blocks)
if pageIdentity != "" && seenPageIdentities[pageIdentity] {
return nil, fmt.Errorf("list_document_blocks 分页停滞,无法证明回读完整")
}
if pageIdentity != "" {
seenPageIdentities[pageIdentity] = true
}
all = append(all, blocks...)
hasMore, known, _ := docPageState(page)
if known && !hasMore {
return map[string]any{"blocks": all, "hasMore": false, "totalCount": len(all)}, nil
}
if !known {
if total, ok := nestedNonNegativeInt(page, "totalCount", "total_count"); ok && len(all) >= total {
return map[string]any{"blocks": all, "hasMore": false, "totalCount": total}, nil
}
}
if !known && len(blocks) < docBlockReadPageSize {
return map[string]any{"blocks": all, "hasMore": false, "totalCount": len(all)}, nil
}
if len(blocks) == 0 {
return nil, fmt.Errorf("list_document_blocks 声明仍有下一页但当前页为空,无法证明回读完整")
}
}
return nil, fmt.Errorf("list_document_blocks 超过 %d 个块,无法在安全上限内完成回读", docBlockReadMaxItems)
}
func documentBlockPageIdentity(blocks []any) string {
if len(blocks) == 0 {
return ""
}
ids := make([]string, 0, len(blocks))
for _, value := range blocks {
id := ""
switch block := value.(type) {
case map[string]any:
id = blockIdentity(block, "")
if id == "" {
if element, ok := block["element"].(map[string]any); ok {
id = blockIdentity(element, "")
}
}
case []any:
id = jsonMLBlockIdentity(block)
}
if id == "" {
return ""
}
ids = append(ids, id)
}
encoded, _ := json.Marshal(ids)
return string(encoded)
}
func documentBlockEntries(value any) ([]any, bool) {
switch typed := value.(type) {
case map[string]any:
for _, key := range []string{"blocks", "items"} {
if blocks, ok := typed[key].([]any); ok {
return blocks, true
}
}
if encoded, ok := typed["jsonml"].(string); ok {
var decoded any
if json.Unmarshal([]byte(encoded), &decoded) == nil {
blocks := orderedJSONMLBlocks(decoded)
values := make([]any, len(blocks))
for index := range blocks {
values[index] = blocks[index]
}
return values, true
}
}
for _, key := range []string{"result", "data"} {
if nested, ok := typed[key]; ok {
if blocks, found := documentBlockEntries(nested); found {
return blocks, true
}
}
}
}
return nil, false
}
func nestedNonNegativeInt(value any, keys ...string) (int, bool) {
switch typed := value.(type) {
case map[string]any:
for _, key := range keys {
if raw, ok := typed[key]; ok {
switch number := raw.(type) {
case float64:
if number >= 0 && number == float64(int(number)) {
return int(number), true
}
case int:
if number >= 0 {
return number, true
}
}
}
}
for _, key := range []string{"result", "data"} {
if result, ok := nestedNonNegativeInt(typed[key], keys...); ok {
return result, true
}
}
}
return 0, false
}
func splitDocMarkdown(content string, maxRunes int) []string {
if maxRunes <= 0 {
return []string{content}
@@ -796,11 +1000,17 @@ func containsText(value any, needle string) bool {
}
func verifyUpdatedDocumentContent(value any, expected, mode, format string) bool {
expected = normalizeDocumentContentForVerification(expected, format)
expectedRaw := expected
expected = normalizeDocumentContentForVerification(expectedRaw, format)
for _, candidate := range documentContentCandidates(value, format) {
actual := normalizeDocumentContentForVerification(candidate, format)
actualRaw := candidate
actual := normalizeDocumentContentForVerification(actualRaw, format)
if mode == "overwrite" {
if actual == expected {
if actual == expected || (format == "markdown" && stripReadbackDocumentTitle(actual) == expected) {
return true
}
if format == "markdown" && (markdownSemanticallyEquivalent(actualRaw, expectedRaw) ||
markdownSemanticallyEquivalent(stripReadbackDocumentTitle(actualRaw), expectedRaw)) {
return true
}
continue
@@ -808,10 +1018,48 @@ func verifyUpdatedDocumentContent(value any, expected, mode, format string) bool
if actual == expected || strings.HasSuffix(actual, "\n"+expected) {
return true
}
if format == "markdown" && markdownSemanticallyEndsWith(actualRaw, expectedRaw) {
return true
}
}
return false
}
func markdownSemanticallyEquivalent(left, right string) bool {
leftFingerprint, leftOK := markdownSemanticFingerprint(left)
rightFingerprint, rightOK := markdownSemanticFingerprint(right)
return leftOK && rightOK && leftFingerprint == rightFingerprint
}
func markdownSemanticallyEndsWith(content, suffix string) bool {
contentFingerprint, contentOK := markdownSemanticFingerprint(content)
suffixFingerprint, suffixOK := markdownSemanticFingerprint(suffix)
return contentOK && suffixOK && strings.HasSuffix(contentFingerprint, suffixFingerprint)
}
func markdownSemanticFingerprint(source string) (string, bool) {
if len(source) > docMarkdownVerifyMax {
return "", false
}
var rendered bytes.Buffer
if err := docMarkdownConvert([]byte(source), &rendered); err != nil {
return "", false
}
return rendered.String(), true
}
func stripReadbackDocumentTitle(content string) string {
lines := strings.Split(content, "\n")
if len(lines) == 0 || !strings.HasPrefix(strings.TrimSpace(lines[0]), "# ") {
return content
}
lines = lines[1:]
for len(lines) > 0 && strings.TrimSpace(lines[0]) == "" {
lines = lines[1:]
}
return strings.Join(lines, "\n")
}
func verifyInsertedBlock(result, data map[string]any, referenceBlockID, expected, format string) bool {
return verifyInsertedCanonicalBlock(result, data, referenceBlockID, normalizeDocumentContentForVerification(expected, format), format)
}
@@ -848,6 +1096,11 @@ func blockContentEquals(data map[string]any, blockID, expected, format string) b
}
func canonicalBlockContent(value any, format string) string {
if values, ok := value.(map[string]any); ok {
if element, ok := values["element"].(map[string]any); ok {
value = element
}
}
if format == "jsonml" {
if values, ok := value.(map[string]any); ok {
if encoded, ok := values["jsonml"].(string); ok {
@@ -871,7 +1124,7 @@ func canonicalBlockContent(value any, format string) string {
return
}
for key, child := range typed {
if key == "id" || key == "blockId" || key == "uuid" {
if key == "id" || key == "blockId" || key == "uuid" || key == "blockType" {
continue
}
walk(child)
@@ -1005,6 +1258,10 @@ func orderedDocumentBlocks(value any) []map[string]any {
walk = func(current any) {
switch typed := current.(type) {
case map[string]any:
if element, ok := typed["element"].(map[string]any); ok && blockIdentity(element, "") != "" {
blocks = append(blocks, element)
return
}
if blockIdentity(typed, "") != "" {
blocks = append(blocks, typed)
return
@@ -1065,9 +1322,23 @@ func normalizeDocumentContentForVerification(raw, format string) string {
func normalizeMarkdownForVerification(raw string) string {
raw = strings.ReplaceAll(strings.ReplaceAll(raw, "\r\n", "\n"), "\r", "\n")
lines := make([]string, 0, strings.Count(raw, "\n")+1)
inFence := false
for _, line := range strings.Split(raw, "\n") {
line = strings.TrimSpace(line)
trimmed := strings.TrimSpace(line)
if strings.HasPrefix(trimmed, "```") || strings.HasPrefix(trimmed, "~~~") {
inFence = !inFence
lines = append(lines, trimmed)
continue
}
if inFence {
lines = append(lines, strings.TrimRight(line, " \t"))
continue
}
line = trimmed
if line == "" {
if len(lines) > 0 && lines[len(lines)-1] != "" {
lines = append(lines, "")
}
continue
}
if strings.Contains(line, "|") {
@@ -1081,6 +1352,9 @@ func normalizeMarkdownForVerification(raw string) string {
}
lines = append(lines, line)
}
for len(lines) > 0 && lines[len(lines)-1] == "" {
lines = lines[:len(lines)-1]
}
return strings.Join(lines, "\n")
}
@@ -1089,33 +1363,108 @@ func normalizeJSONMLForVerification(raw string) string {
if err := json.Unmarshal([]byte(raw), &value); err != nil {
return normalizeMarkdownForVerification(raw)
}
var tokens []string
var walk func(any)
walk = func(current any) {
var normalize func(any) any
normalize = func(current any) any {
switch typed := current.(type) {
case []any:
start := 0
if len(typed) > 0 {
if tag, ok := typed[0].(string); ok {
tokens = append(tokens, "<"+strings.ToLower(strings.TrimSpace(tag))+">")
start = 1
if len(typed) == 0 {
return []any{}
}
tag, isElement := typed[0].(string)
if !isElement {
out := make([]any, 0, len(typed))
for _, child := range typed {
out = append(out, normalize(child))
}
return out
}
start := 1
attrs := map[string]any{}
if len(typed) > 1 {
if declared, ok := typed[1].(map[string]any); ok {
attrs, _ = normalize(declared).(map[string]any)
attrs = removeGeneratedJSONMLDefaults(tag, attrs)
start = 2
}
}
children := make([]any, 0, len(typed)-start)
for _, child := range typed[start:] {
walk(child)
normalized := normalize(child)
if normalized != nil {
children = append(children, normalized)
}
}
if strings.EqualFold(tag, "span") && isGeneratedTextSpan(attrs) {
if len(children) == 1 {
return children[0]
}
return children
}
out := []any{strings.ToLower(tag), attrs}
out = append(out, children...)
return out
case map[string]any:
// JSONML maps contain element attributes. Server-generated UUIDs and
// default attributes do not change the authored document content.
return
case string:
if text := normalizeMarkdownForVerification(typed); text != "" {
tokens = append(tokens, text)
out := make(map[string]any, len(typed))
for key, child := range typed {
normalizedKey := strings.ToLower(strings.NewReplacer("_", "", "-", "").Replace(key))
if normalizedKey == "uuid" || normalizedKey == "blockid" || normalizedKey == "elementid" || normalizedKey == "index" {
continue
}
out[normalizedKey] = normalize(child)
}
return out
case string:
return strings.ReplaceAll(strings.ReplaceAll(typed, "\r\n", "\n"), "\r", "\n")
}
return current
}
walk(value)
return strings.Join(tokens, "\n")
// normalize only receives values decoded by encoding/json, so the resulting
// tree is always JSON-marshalable.
encoded, _ := json.Marshal(normalize(value))
return string(encoded)
}
var generatedJSONMLAttributeDefaults = map[string]map[string]any{
"hr": {
"sz": float64(1),
},
"tc": {
"colspan": float64(1), "rowspan": float64(1), "valign": "middle",
},
"code": {
"code": "", "syntax": "plaintext", "theme": "default",
"wrap": true, "showlinenumber": true, "fold": false,
},
}
// removeGeneratedJSONMLDefaults drops only defaults declared by the reviewed
// JSONML schema, plus empty server style objects. Other attributes remain part
// of the semantic fingerprint so links, formatting, and table layout stay
// strict.
func removeGeneratedJSONMLDefaults(tag string, attrs map[string]any) map[string]any {
defaults := generatedJSONMLAttributeDefaults[strings.ToLower(tag)]
out := make(map[string]any, len(attrs))
for key, value := range attrs {
if object, ok := value.(map[string]any); ok && len(object) == 0 {
continue
}
if defaultValue, ok := defaults[key]; ok && reflect.DeepEqual(value, defaultValue) {
continue
}
out[key] = value
}
return out
}
func isGeneratedTextSpan(attrs map[string]any) bool {
if len(attrs) == 0 {
return true
}
if len(attrs) != 1 {
return false
}
value, ok := attrs["datatype"].(string)
return ok && (value == "text" || value == "leaf")
}
func executeExport(rt *shortcut.RuntimeContext) error {
@@ -0,0 +1,401 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0
package doc
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"strings"
"testing"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/helpers"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/shortcut"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/testseam"
"github.com/spf13/cobra"
)
func TestCrossPlatformCoverageDocReadbackRetriesStaleContent(t *testing.T) {
testseam.Swap(t, &docVerifyWait, func(context.Context, time.Duration) error { return nil })
testseam.Swap(t, &docVerifyDelays, []time.Duration{time.Millisecond})
caller := &docCoverageCaller{responses: map[string][]map[string]any{
"get_document_content": {{"markdown": "old"}, {"markdown": "old\nnew"}},
}}
if err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "append", "--content", "new", "--yes"); err != nil {
t.Fatal(err)
}
reads := 0
for _, call := range caller.history {
if call.tool == "get_document_content" {
reads++
}
}
if reads != 2 {
t.Fatalf("readback calls = %d, want 2; history=%#v", reads, caller.history)
}
}
func TestCrossPlatformCoverageDocReadbackStopsOnCancellation(t *testing.T) {
if err := waitForDocVerification(nil, time.Nanosecond); err != nil {
t.Fatalf("completed verification wait = %v", err)
}
cancelled, cancel := context.WithCancel(context.Background())
cancel()
if err := waitForDocVerification(cancelled, time.Hour); !errors.Is(err, context.Canceled) {
t.Fatalf("cancelled verification wait = %v, want context.Canceled", err)
}
cmd := &cobra.Command{Use: "verify"}
cmd.SetContext(cancelled)
caller := &docCoverageCaller{responses: map[string][]map[string]any{
"get_document_content": {{"markdown": "stale"}},
}}
helpers.InitDeps(caller)
rt := shortcut.RuntimeContextForTest(cmd, Update)
_, err := readDocVerification(rt, "get_document_content", map[string]any{"nodeId": "n"}, func(map[string]any) bool { return false })
if !errors.Is(err, context.Canceled) {
t.Fatalf("cancelled document verification = %v, want context.Canceled", err)
}
if len(caller.history) != 1 {
t.Fatalf("cancelled document verification calls = %d, want 1", len(caller.history))
}
}
func TestCrossPlatformCoverageDocDeleteReadbackConsumesEveryPage(t *testing.T) {
testseam.Swap(t, &docVerifyWait, func(context.Context, time.Duration) error { return nil })
testseam.Swap(t, &docVerifyDelays, []time.Duration{time.Millisecond})
firstPage := make([]any, 50)
for index := range firstPage {
firstPage[index] = map[string]any{"id": fmt.Sprintf("block-%d", index), "text": "body"}
}
caller := &docCoverageCaller{responses: map[string][]map[string]any{
"list_document_blocks": {
{"blocks": firstPage, "hasMore": true, "totalCount": 51},
{"blocks": []any{map[string]any{"id": "target", "text": "stale"}}, "hasMore": false, "totalCount": 51},
{"blocks": firstPage, "hasMore": false, "totalCount": 50},
},
}}
if err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "block_delete", "--block-id", "target", "--yes"); err != nil {
t.Fatal(err)
}
starts := []int{}
for _, call := range caller.history {
if call.tool == "list_document_blocks" {
starts = append(starts, call.params["startIndex"].(int))
}
}
if fmt.Sprint(starts) != "[0 50 0]" {
t.Fatalf("pagination starts = %v, want [0 50 0]", starts)
}
}
func TestCrossPlatformCoverageDocReplacePreflightIsGloballyUnique(t *testing.T) {
firstPage := make([]any, 50)
for index := range firstPage {
text := "body"
if index == 0 {
text = "unique needle"
}
firstPage[index] = map[string]any{"id": fmt.Sprintf("block-%d", index), "text": text}
}
caller := &docCoverageCaller{responses: map[string][]map[string]any{
"list_document_blocks": {
{"blocks": firstPage, "hasMore": true, "totalCount": 51},
{"blocks": []any{map[string]any{"id": "block-50", "text": "another needle"}}, "hasMore": false, "totalCount": 51},
},
}}
err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "str_replace", "--old", "needle", "--new", "changed", "--yes")
if err == nil {
t.Fatal("str_replace accepted a second match on a later page")
}
for _, call := range caller.history {
if call.tool == "update_document_block" {
t.Fatalf("ambiguous replace executed a write: %#v", caller.history)
}
}
}
func TestCrossPlatformCoverageDocVerificationPreservesMeaning(t *testing.T) {
expected := "[\"p\",{},\"text\"]"
serverExpanded := "[\"p\",{\"uuid\":\"generated\",\"style\":{}},[\"span\",{\"data-type\":\"text\"},[\"span\",{\"data-type\":\"leaf\"},\"text\"]]]"
if normalizeJSONMLForVerification(expected) != normalizeJSONMLForVerification(serverExpanded) {
t.Fatal("generated JSONML text wrappers should not change document meaning")
}
defaultFree := `[["hr",{}],["code",{}],["table",{},["tr",{},["tc",{},["p",{},"cell"]]]]]`
serverDefaulted := `[["hr",{"sz":1}],["code",{"code":"","syntax":"plaintext","theme":"default","wrap":true,"showLineNumber":true,"fold":false}],["table",{},["tr",{},["tc",{"colSpan":1,"rowSpan":1,"vAlign":"middle"},["p",{},"cell"]]]]]`
if !verifyUpdatedDocumentContent(map[string]any{"jsonml": serverDefaulted}, defaultFree, "overwrite", "jsonml") {
t.Fatal("server-generated JSONML schema defaults should not fail readback verification")
}
linkA := "[\"a\",{\"href\":\"https://example.com/a\"},\"text\"]"
linkB := "[\"a\",{\"href\":\"https://example.com/b\"},\"text\"]"
if normalizeJSONMLForVerification(linkA) == normalizeJSONMLForVerification(linkB) {
t.Fatal("semantic JSONML attributes were ignored")
}
tableA := `[["table",{"jc":"center"},["tr",{},["tc",{},["p",{},"cell"]]]]]`
tableB := `[["table",{"jc":"right"},["tr",{},["tc",{},["p",{},"cell"]]]]]`
if normalizeJSONMLForVerification(tableA) == normalizeJSONMLForVerification(tableB) {
t.Fatal("semantic JSONML table alignment was ignored")
}
codeA := "~~~go\n return nil\n~~~"
codeB := "~~~go\nreturn nil\n~~~"
if normalizeMarkdownForVerification(codeA) == normalizeMarkdownForVerification(codeB) {
t.Fatal("fenced code indentation was ignored")
}
if !verifyUpdatedDocumentContent(map[string]any{"markdown": "# Server title\n\nbody"}, "body", "overwrite", "markdown") {
t.Fatal("server-generated document title prevented body verification")
}
}
func TestCrossPlatformCoverageMarkdownSemanticRoundTrip(t *testing.T) {
input := strings.Join([]string{
"sales_data.xlsx",
"### 1.",
"+10.22%",
"| name | value |",
"| -------- | -------- |",
"| sales_data.xlsx | +10.22% |",
}, "\n")
server := strings.Join([]string{
`sales\_data.xlsx`,
`### 1\.`,
`\+10.22%`,
"|name|value|",
"|---|---|",
`|sales\_data.xlsx|\+10.22%|`,
}, "\n")
if !markdownSemanticallyEquivalent(input, server) {
t.Fatal("server Markdown escaping and table delimiter normalization changed the semantic fingerprint")
}
if !verifyUpdatedDocumentContent(map[string]any{"markdown": server}, input, "overwrite", "markdown") {
t.Fatal("equivalent server Markdown failed overwrite verification")
}
if !verifyUpdatedDocumentContent(map[string]any{"markdown": "existing\n\n" + server}, input, "append", "markdown") {
t.Fatal("equivalent server Markdown failed append verification")
}
}
func TestCrossPlatformCoverageMarkdownSemanticDifferencesRemainStrict(t *testing.T) {
for _, test := range []struct {
name string
left string
right string
}{
{name: "emphasis", left: `*important*`, right: `\*important\*`},
{name: "inline code", left: "`sales_data`", right: "`sales\\_data`"},
{name: "fenced code", left: "```\nsales_data\n```", right: "```\nsales\\_data\n```"},
{name: "table alignment", left: "|a|\n|---|\n|x|", right: "|a|\n|:---|\n|x|"},
{name: "table columns", left: "|a|b|\n|---|---|\n|x|y|", right: "|a|\n|---|\n|x|"},
{name: "table content", left: "|a|\n|---|\n|x|", right: "|a|\n|---|\n|y|"},
} {
t.Run(test.name, func(t *testing.T) {
if markdownSemanticallyEquivalent(test.left, test.right) {
t.Fatal("meaningful Markdown difference was ignored")
}
})
}
oversized := strings.Repeat("x", docMarkdownVerifyMax+1)
if _, ok := markdownSemanticFingerprint(oversized); ok {
t.Fatal("oversized Markdown entered semantic verification")
}
testseam.Swap(t, &docMarkdownConvert, func([]byte, io.Writer) error { return errors.New("render") })
if _, ok := markdownSemanticFingerprint("body"); ok {
t.Fatal("failed Markdown render produced a semantic fingerprint")
}
}
func TestCrossPlatformCoverageDocElementReadbackUsesNestedElement(t *testing.T) {
wrapper := map[string]any{
"blockType": "paragraph",
"element": map[string]any{"id": "inserted", "blockType": "paragraph", "paragraph": map[string]any{"text": "body"}},
}
if got := canonicalBlockContent(wrapper, "markdown"); got != "body" {
t.Fatalf("nested element content = %q, want body", got)
}
blocks := orderedDocumentBlocks(map[string]any{"blocks": []any{wrapper}})
if len(blocks) != 1 || blockIdentity(blocks[0], "") != "inserted" {
t.Fatalf("nested element blocks = %#v", blocks)
}
}
func TestCrossPlatformCoverageVersionRevertRequiresTargetEvidence(t *testing.T) {
if revertResultMatchesVersion(map[string]any{"ok": true}, 3) || currentDocumentMatchesRestoredVersion(map[string]any{"version": 99}, 3) {
t.Fatal("readability or an unrelated current version must not prove a revert")
}
if revertResultMatchesVersion(map[string]any{"version": 3}, 3) {
t.Fatal("the request version parameter must not prove its own revert")
}
if revertResultMatchesVersion(map[string]any{"data": map[string]any{"request": map[string]any{"targetVersion": 3}}}, 3) {
t.Fatal("request echo containers must not provide version evidence")
}
for _, failed := range []map[string]any{
{"data": map[string]any{"success": "false", "revertedToVersion": 3}},
{"data": map[string]any{"ok": false, "revertedToVersion": 3}},
{"data": map[string]any{"status": "FAILED", "revertedToVersion": 3}},
{"data": map[string]any{"state": "failure", "revertedToVersion": 3}},
{"data": []any{map[string]any{"error_code": "REVERT_FAILED", "revertedToVersion": 3}}},
{"data": map[string]any{"errorCode": 500.0, "revertedToVersion": 3}},
{"data": map[string]any{"code": json.Number("500"), "revertedToVersion": 3}},
} {
if revertResultMatchesVersion(failed, 3) {
t.Fatalf("explicit failure %#v must override target-version evidence", failed)
}
}
for _, succeeded := range []map[string]any{
{"revertedToVersion": 3},
{"status": "SUCCESS", "revertedToVersion": 3},
{"state": "succeeded", "revertedToVersion": 3},
{"errorCode": "0", "revertedToVersion": 3},
{"code": 200, "revertedToVersion": 3},
{"code": "204", "revertedToVersion": 3},
} {
if !revertResultMatchesVersion(succeeded, 3) {
t.Fatalf("explicit success %#v suppressed target-version evidence", succeeded)
}
}
for _, absentOrSuccess := range []any{nil, "", "OK", "SUCCESS", json.Number("0"), json.Number("200"), 0.0, 201.0, 0, 202} {
if revertErrorCodeIsFailure(absentOrSuccess) {
t.Fatalf("success code %#v was treated as an explicit failure", absentOrSuccess)
}
}
for _, failedCode := range []any{json.Number("bad"), json.Number("500"), 1.0, 500.0, 1, 500} {
if !revertErrorCodeIsFailure(failedCode) {
t.Fatalf("failure code %#v was not treated as an explicit failure", failedCode)
}
}
if revertStatusIsFailure(500) || revertStatusIsFailure("PROCESSING") {
t.Fatal("non-failure status was treated as an explicit failure")
}
}
func TestCrossPlatformCoverageDocReadbackDefensiveEdges(t *testing.T) {
testseam.Swap(t, &docVerifyWait, func(context.Context, time.Duration) error { return nil })
for _, tc := range []struct {
name string
responses []map[string]any
failAt int
}{
{"call failure", nil, 1},
{"missing blocks", []map[string]any{{"ok": true}}, 0},
{"stalled page", []map[string]any{{"blocks": []any{map[string]any{"id": "a"}}, "hasMore": true}, {"blocks": []any{map[string]any{"id": "a"}}, "hasMore": true}}, 0},
{"empty continued page", []map[string]any{{"blocks": []any{}, "hasMore": true}}, 0},
} {
t.Run(tc.name, func(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
caller := &docCoverageCaller{failAt: tc.failAt, responses: map[string][]map[string]any{"list_document_blocks": tc.responses}}
err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "block_delete", "--block-id", "target", "--yes")
if err == nil {
t.Fatal("defensive readback unexpectedly succeeded")
}
})
}
t.Run("identical adjacent pages advance by requested indexes", func(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
blocks := make([]any, docBlockReadPageSize)
for index := range blocks {
blocks[index] = map[string]any{"blockType": "paragraph"}
}
caller := &docCoverageCaller{responses: map[string][]map[string]any{"list_document_blocks": {
{"blocks": blocks, "hasMore": true, "totalCount": 2 * docBlockReadPageSize},
{"blocks": blocks, "hasMore": false, "totalCount": 2 * docBlockReadPageSize},
}}}
if err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "block_delete", "--block-id", "target", "--yes"); err != nil {
t.Fatal(err)
}
starts := []int{}
for _, call := range caller.history {
if call.tool == "list_document_blocks" {
starts = append(starts, call.params["startIndex"].(int))
}
}
if fmt.Sprint(starts) != "[0 50]" {
t.Fatalf("pagination starts = %v, want [0 50]", starts)
}
})
t.Run("total count terminates pagination", func(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
caller := &docCoverageCaller{responses: map[string][]map[string]any{"list_document_blocks": {{"blocks": []any{map[string]any{"id": "other"}}, "totalCount": 1}}}}
if err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "block_delete", "--block-id", "target", "--yes"); err != nil {
t.Fatal(err)
}
})
t.Run("explicit has more overrides inconsistent total count", func(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
caller := &docCoverageCaller{responses: map[string][]map[string]any{"list_document_blocks": {
{"blocks": []any{map[string]any{"id": "first"}}, "hasMore": true, "totalCount": 1},
{"blocks": []any{map[string]any{"id": "second"}}, "hasMore": false, "totalCount": 2},
}}}
if err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "block_delete", "--block-id", "target", "--yes"); err != nil {
t.Fatal(err)
}
reads := 0
for _, call := range caller.history {
if call.tool == "list_document_blocks" {
reads++
}
}
if reads != 2 {
t.Fatalf("read calls = %d, want both explicitly advertised pages", reads)
}
})
t.Run("block read safety limit", func(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
pages := make([]map[string]any, docBlockReadMaxItems/docBlockReadPageSize)
for index := range pages {
pages[index] = map[string]any{"blocks": []any{map[string]any{"id": fmt.Sprintf("block-%d", index)}}, "hasMore": true}
}
caller := &docCoverageCaller{responses: map[string][]map[string]any{"list_document_blocks": pages}}
if err := runDocCoverage(t, Update, caller, "--node", "n", "--command", "block_delete", "--block-id", "target", "--yes"); err == nil {
t.Fatal("oversized block read returned nil")
}
})
if blocks, ok := documentBlockEntries(map[string]any{"jsonml": `["root",{},["p",{"uuid":"a"},"x"]]`}); !ok || len(blocks) == 0 {
t.Fatalf("jsonml blocks=%#v ok=%v", blocks, ok)
}
if _, ok := documentBlockEntries(map[string]any{"jsonml": `{`}); ok {
t.Fatal("invalid jsonml produced blocks")
}
if blocks, ok := documentBlockEntries(map[string]any{"data": map[string]any{"items": []any{"x"}}}); !ok || len(blocks) != 1 {
t.Fatalf("nested items=%#v ok=%v", blocks, ok)
}
if _, ok := documentBlockEntries(nil); ok {
t.Fatal("nil produced blocks")
}
for _, tc := range []struct {
value any
want bool
}{
{map[string]any{"totalCount": float64(2)}, true},
{map[string]any{"totalCount": float64(-1)}, false},
{map[string]any{"totalCount": 2.5}, false},
{map[string]any{"data": map[string]any{"total_count": 2}}, true},
{map[string]any{"totalCount": -1}, false},
{nil, false},
} {
_, ok := nestedNonNegativeInt(tc.value, "totalCount", "total_count")
if ok != tc.want {
t.Fatalf("nestedNonNegativeInt(%#v) ok=%v want=%v", tc.value, ok, tc.want)
}
}
for _, raw := range []string{
`[]`, `[1,["p",{},"x"]]`, `["span",{},"a","b"]`,
`["p",{"block_id":"x","custom":true},"x"]`, `true`,
} {
if normalizeJSONMLForVerification(raw) == "" {
t.Fatalf("empty normalized JSONML for %s", raw)
}
}
if !isGeneratedTextSpan(nil) || isGeneratedTextSpan(map[string]any{"a": 1, "b": 2}) || isGeneratedTextSpan(map[string]any{"data-type": 3}) || isGeneratedTextSpan(map[string]any{"data-type": "other"}) {
t.Fatal("generated span classification failed")
}
}
+55 -3
View File
@@ -15,6 +15,7 @@ import (
"strings"
"sync"
"testing"
"time"
"unicode/utf8"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/corecmd"
@@ -57,7 +58,7 @@ func (f *docCoverageCaller) CallTool(_ context.Context, _, tool string, params m
}
value := docCoveragePayload(tool)
if tool == "revert_doc_version" {
value = map[string]any{"version": params["version"]}
value = map[string]any{"revertedToVersion": params["version"]}
}
if queue := f.responses[tool]; len(queue) > 0 {
value = queue[0]
@@ -112,6 +113,7 @@ func runDocCoverageInput(t *testing.T, declaration shortcut.Shortcut, caller *do
func runDocCoveragePath(t *testing.T, declaration shortcut.Shortcut, caller *docCoverageCaller, input io.Reader, commandPath string, args ...string) error {
t.Helper()
testseam.Swap(t, &docVerifyWait, func(context.Context, time.Duration) error { return nil })
helpers.InitDeps(caller)
root := &cobra.Command{Use: "dws", SilenceErrors: true, SilenceUsage: true}
root.PersistentFlags().Bool("yes", false, "")
@@ -491,6 +493,7 @@ func TestCrossPlatformCoverageDocUpdateAliasReachesNestedBranches(t *testing.T)
}
func TestCrossPlatformCoverageDocWritesStopOnUnknownCommitAndRequireVerification(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
unknown := &docCoverageCaller{failAt: 1, responses: map[string][]map[string]any{}}
err := runDocCoverage(t, Update, unknown, "--node", "n", "--command", "append", "--content", "x", "--yes")
var typed *apperrors.Error
@@ -512,6 +515,7 @@ func TestCrossPlatformCoverageDocWritesStopOnUnknownCommitAndRequireVerification
}
func TestCrossPlatformCoverageDocCreateRejectsSuccessfulMismatchedReadback(t *testing.T) {
testseam.Swap(t, &docVerifyDelays, []time.Duration{})
caller := &docCoverageCaller{responses: map[string][]map[string]any{
"get_document_content": {{"markdown": "truncated"}},
}}
@@ -577,11 +581,59 @@ func TestCrossPlatformCoverageDocVersionRevertPaginationAndVerification(t *testi
"revert_doc_version": {{}},
"get_document_info": {{"nodeId": "n", "revision": 99.0}},
}}
if err := runDocCoverage(t, VersionRevert, caller, "--node", "n", "--version", "3", "--yes"); err != nil {
t.Fatal(err)
err := runDocCoverage(t, VersionRevert, caller, "--node", "n", "--version", "3", "--yes")
var typed *apperrors.Error
if !errors.As(err, &typed) || typed.Reason != "doc_history_revert_target_unproven" || typed.FailureStage != "verify" || typed.Details["status"] != "partial_success" {
t.Fatalf("unproven revert error = %#v", err)
}
data, _ := typed.Details["data"].(map[string]any)
steps, _ := typed.Details["steps"].([]map[string]any)
if data["verified"] != false || len(steps) != 3 || steps[2]["status"] != "failed" {
t.Fatalf("unproven revert details = %#v", typed.Details)
}
})
for _, test := range []struct {
name string
response map[string]any
current map[string]any
wantOK bool
}{
{name: "explicit server evidence", response: map[string]any{"data": map[string]any{"revertResult": map[string]any{"revertedToVersion": 3}}}, wantOK: true},
{name: "bare request parameter echo", response: map[string]any{"version": 3}},
{name: "nested request parameter echo", response: map[string]any{"data": map[string]any{"request": map[string]any{"nodeId": "n", "version": 3}}}},
{name: "accepted field inside request echo", response: map[string]any{"data": map[string]any{"request": map[string]any{"targetVersion": 3}}}},
{name: "nested business failure with request echo", response: map[string]any{"data": map[string]any{"success": false, "errorCode": "REVERT_FAILED", "request": map[string]any{"version": 3}}}},
{name: "failed status overrides target evidence", response: map[string]any{"data": map[string]any{"status": "FAILED", "revertResult": map[string]any{"revertedToVersion": 3}}}},
{name: "failed code overrides target evidence", response: map[string]any{"data": map[string]any{"code": "REVERT_FAILED", "revertResult": map[string]any{"revertedToVersion": 3}}}},
{
name: "nested business failure overrides all evidence",
response: map[string]any{"data": map[string]any{"state": "FAILURE", "revertResult": map[string]any{"revertedToVersion": 3}}},
current: map[string]any{"targetVersion": 3},
},
} {
t.Run(test.name, func(t *testing.T) {
responses := map[string][]map[string]any{
"revert_doc_version": {test.response},
}
if test.current != nil {
responses["get_document_info"] = []map[string]any{test.current}
}
caller := &docCoverageCaller{responses: responses}
err := runDocCoverage(t, VersionRevert, caller, "--node", "n", "--version", "3", "--yes")
if test.wantOK {
if err != nil {
t.Fatal(err)
}
return
}
var typed *apperrors.Error
if !errors.As(err, &typed) || typed.Reason != "doc_history_revert_target_unproven" || typed.Details["status"] != "partial_success" {
t.Fatalf("request echo response %#v produced %#v", test.response, err)
}
})
}
for _, test := range []struct {
name string
responses []map[string]any
@@ -178,15 +178,51 @@ func executeHistoryRevert(rt *shortcut.RuntimeContext) error {
map[string]any{"available": false, "reason": "the requested revert completed; verify the current document before any further write"},
)
}
verified := !revertResponseHasExplicitFailure(reverted) &&
(revertResultMatchesVersion(reverted, target) || currentDocumentMatchesRestoredVersion(current, target))
if !verified {
return docPartialWriteError(
"doc.history_revert", "doc_history_revert_target_unproven", "verify",
fmt.Sprintf("版本 %d 的回滚请求已执行且文档可读,但响应没有提供目标版本证据;不要直接重试回滚", target),
fmt.Errorf("回读缺少目标版本 %d 的明确证据", target),
map[string]any{
"nodeId": nodeID, "version": target, "reverted": true, "verified": false,
"revertResult": reverted, "current": current,
},
[]map[string]any{
{"name": "preflight", "status": "success"},
{"name": "revert", "status": "success"},
{"name": "verify", "status": "failed"},
},
map[string]any{"available": false, "reason": "the revert may have completed; inspect version history before any further revert"},
)
}
return rt.Output(docEnvelope("doc.history_revert", map[string]any{
"version": target, "revertResult": reverted, "current": current, "verified": true,
"verification": "revert_acknowledged_and_document_readable",
"verification": "target_version_proven",
},
map[string]any{"name": "preflight", "status": "success"},
map[string]any{"name": "revert", "status": "success"},
map[string]any{"name": "verify", "status": "success"}))
}
func revertResultMatchesVersion(value map[string]any, target int) bool {
if revertResponseHasExplicitFailure(value) {
return false
}
return versionEvidenceMatches(value, target, map[string]bool{
"targetversion": true, "appliedversion": true,
"restoredversion": true, "revertedversion": true, "revertedtoversion": true, "sourceversion": true,
})
}
func currentDocumentMatchesRestoredVersion(value map[string]any, target int) bool {
return versionEvidenceMatches(value, target, map[string]bool{
"restoredfromversion": true, "revertedfromversion": true,
"sourceversion": true, "appliedversion": true, "targetversion": true,
})
}
func findHistoryVersion(rt *shortcut.RuntimeContext, nodeID string, target int) (bool, error) {
const maxPages = 20
cursor := ""
@@ -240,6 +276,9 @@ func versionEvidenceMatches(value any, target int, acceptedKeys map[string]bool)
case map[string]any:
for key, child := range typed {
normalized := strings.ToLower(strings.NewReplacer("_", "", "-", "").Replace(key))
if versionEvidenceRequestEchoKeys[normalized] {
continue
}
if acceptedKeys[normalized] {
if versionNumberMatches(child, target) {
return true
@@ -259,6 +298,87 @@ func versionEvidenceMatches(value any, target int, acceptedKeys map[string]bool)
return false
}
var versionEvidenceRequestEchoKeys = map[string]bool{
"args": true, "arguments": true, "input": true, "inputs": true,
"params": true, "parameters": true, "request": true, "requestbody": true,
"requestparams": true, "toolargs": true, "toolarguments": true,
}
func revertResponseHasExplicitFailure(value any) bool {
switch typed := value.(type) {
case map[string]any:
for key, child := range typed {
normalized := strings.ToLower(strings.NewReplacer("_", "", "-", "").Replace(key))
if versionEvidenceRequestEchoKeys[normalized] {
continue
}
if normalized == "success" || normalized == "ok" {
if success, ok := child.(bool); ok && !success {
return true
}
if success, ok := child.(string); ok && strings.EqualFold(strings.TrimSpace(success), "false") {
return true
}
}
if (normalized == "status" || normalized == "state") && revertStatusIsFailure(child) {
return true
}
if normalized == "errorcode" || normalized == "code" {
if revertErrorCodeIsFailure(child) {
return true
}
}
if revertResponseHasExplicitFailure(child) {
return true
}
}
case []any:
for _, child := range typed {
if revertResponseHasExplicitFailure(child) {
return true
}
}
}
return false
}
func revertStatusIsFailure(value any) bool {
status, ok := value.(string)
if !ok {
return false
}
switch strings.ToLower(strings.TrimSpace(status)) {
case "fail", "failed", "failure", "error", "errored", "reject", "rejected", "deny", "denied", "cancel", "cancelled", "canceled", "abort", "aborted":
return true
default:
return false
}
}
func revertErrorCodeIsFailure(value any) bool {
switch typed := value.(type) {
case string:
normalized := strings.ToLower(strings.TrimSpace(typed))
switch normalized {
case "", "ok", "success", "succeed", "succeeded":
return false
}
if parsed, err := strconv.Atoi(normalized); err == nil {
return parsed != 0 && (parsed < 200 || parsed >= 300)
}
return true
case float64:
return typed != 0 && (typed < 200 || typed >= 300)
case json.Number:
parsed, err := typed.Int64()
return err != nil || (parsed != 0 && (parsed < 200 || parsed >= 300))
case int:
return typed != 0 && (typed < 200 || typed >= 300)
default:
return false
}
}
func versionNumberMatches(value any, target int) bool {
switch number := value.(type) {
case float64:
+3 -7
View File
@@ -618,15 +618,11 @@ func completeMinutesUpload(rt *shortcut.RuntimeContext, sessionID string, timeou
}
message := strings.ToLower(err.Error())
retryable := strings.Contains(message, "still uploading") || strings.Contains(message, "wait and retry") || strings.Contains(message, "正在上传")
if !retryable || time.Now().Add(interval).After(deadline) {
if !retryable || minutesPollDeadlineReached(deadline, interval) {
return nil, attempts, err
}
timer := time.NewTimer(interval)
select {
case <-rt.Command().Context().Done():
timer.Stop()
return nil, attempts, rt.Command().Context().Err()
case <-timer.C:
if err := waitMinutesInterval(rt, interval); err != nil {
return nil, attempts, err
}
}
}
+30 -6
View File
@@ -4,6 +4,7 @@
package minutes
import (
"context"
"encoding/json"
"fmt"
"os"
@@ -372,7 +373,7 @@ func runMinutesMindmap(rt *shortcut.RuntimeContext, id string, timeout, interval
payload := map[string]any{"operation": "minutes.mindmap", "complete": false, "taskUuid": id, "taskStatus": status, "attempts": attempts, "result": result, "recovery": map[string]any{"taskUuid": id, "nextAction": "inspect source transcript; do not assume an empty mind map"}}
return payload, minutesCompositeError("minutes_mindmap_failed", "poll", payload)
}
if time.Now().Add(interval).After(deadline) {
if minutesPollDeadlineReached(deadline, interval) {
payload := map[string]any{"operation": "minutes.mindmap", "complete": false, "taskUuid": id, "taskStatus": status, "attempts": attempts, "recovery": map[string]any{"taskUuid": id, "nextAction": "dws minutes mind-graph status --id <taskUuid>"}}
return payload, minutesCompositeError("minutes_mindmap_timeout", "poll", payload)
}
@@ -422,7 +423,7 @@ func runMinutesSpeakerInsights(rt *shortcut.RuntimeContext, id string, timeout,
payload := map[string]any{"operation": "minutes.speaker_insights", "complete": false, "taskUuid": id, "taskId": taskID, "attempts": attempts, "stage": "poll", "recovery": map[string]any{"taskUuid": id, "taskId": taskID, "nextAction": "dws minutes speaker summary get --ids <taskUuid>"}}
return payload, callErr
}
if time.Now().Add(interval).After(deadline) {
if minutesPollDeadlineReached(deadline, interval) {
payload := map[string]any{"operation": "minutes.speaker_insights", "complete": false, "taskUuid": id, "taskId": taskID, "attempts": attempts, "stage": "poll", "recovery": map[string]any{"taskUuid": id, "taskId": taskID, "nextAction": "dws minutes speaker summary get --ids <taskUuid>"}}
return payload, minutesCompositeError("minutes_speaker_insights_timeout", "poll", payload)
}
@@ -614,6 +615,17 @@ func executeMinutesShare(rt *shortcut.RuntimeContext) error {
}
func executeMinutesUnshare(rt *shortcut.RuntimeContext) error {
if !rt.DryRun() {
for _, id := range minutesIDs(rt) {
data, err := rt.CallMCPData("minutes", "get_minutes_basic_info", map[string]any{"taskUuid": id})
if err != nil {
return fmt.Errorf("minutes unshare preflight for %s: %w", id, err)
}
if _, err := minutesdata.Basic(id, data); err != nil {
return fmt.Errorf("minutes unshare preflight for %s: %w", id, err)
}
}
}
return executeMinutesPermissionLedger(rt, "unshare", "remove_member_permission", func(member string) map[string]any {
return map[string]any{"uuids": minutesIDs(rt), "memberUids": []string{member}}
})
@@ -631,7 +643,11 @@ func executeMinutesPermissionLedger(rt *shortcut.RuntimeContext, operation, tool
for index, member := range members {
data, err := rt.CallMCPWriteDataStrict("minutes", tool, params(member))
if err == nil {
err = minutesdata.RequireWriteAcknowledgement(operation, data)
if operation == "unshare" {
err = minutesdata.RequirePermissionMutationAcknowledgement(operation, minutesIDs(rt), []string{member}, data)
} else {
err = minutesdata.RequireWriteAcknowledgement(operation, data)
}
}
if err != nil {
failures = append(failures, map[string]any{"memberUid": member, "error": err.Error()})
@@ -708,7 +724,7 @@ func waitMinutesArtifacts(rt *shortcut.RuntimeContext, id string, artifacts []st
for {
attempts++
bundle, failures := collectMinutesArtifactsOnce(rt, id, artifacts, pageLimit)
if len(failures) == 0 || time.Now().Add(interval).After(deadline) {
if len(failures) == 0 || minutesPollDeadlineReached(deadline, interval) {
return bundle, failures, attempts
}
if err := waitMinutesInterval(rt, interval); err != nil {
@@ -720,14 +736,22 @@ func waitMinutesArtifacts(rt *shortcut.RuntimeContext, id string, artifacts []st
func waitMinutesInterval(rt *shortcut.RuntimeContext, interval time.Duration) error {
timer := time.NewTimer(interval)
defer timer.Stop()
commandContext := rt.Command().Context()
if commandContext == nil {
commandContext = context.Background()
}
select {
case <-rt.Command().Context().Done():
return rt.Command().Context().Err()
case <-commandContext.Done():
return commandContext.Err()
case <-timer.C:
return nil
}
}
func minutesPollDeadlineReached(deadline time.Time, interval time.Duration) bool {
return !time.Now().Add(interval).Before(deadline)
}
func outputWorkflowResult(rt *shortcut.RuntimeContext, payload map[string]any, failed bool, reason, stage string) error {
if err := rt.Output(payload); err != nil {
return err
@@ -323,7 +323,10 @@ func TestCrossPlatformCoverageMinutesPermissionLedgerBranchesE2E(t *testing.T) {
if err := runMinutesAlignmentCLIWithWriter(t, &minutesE2ECaller{}, minutesFailWriter{}, "minutes", "+unshare", "--id", "u1", "--member-uids", "m1", "--dry-run"); err == nil {
t.Fatal("unshare dry-run output failure accepted")
}
unshare := &minutesE2ECaller{responses: map[string][]string{"minutes/remove_member_permission": {`{"success":true,"result":{}}`}}}
unshare := &minutesE2ECaller{responses: map[string][]string{
"minutes/get_minutes_basic_info": {`{"success":true,"result":{"taskUuid":"u1"}}`},
"minutes/remove_member_permission": {`{"success":true,"result":{"resultMap":{"u1":["m1"]}}}`},
}}
payload, _, err := runMinutesAlignmentCLI(t, unshare, "minutes", "+unshare", "--id", "u1", "--member-uids", "m1", "--yes")
if err != nil || payload["complete"] != true || payload["succeeded"] != float64(1) {
t.Fatalf("unshare payload=%#v err=%v", payload, err)
@@ -337,11 +340,31 @@ func TestCrossPlatformCoverageMinutesPermissionLedgerBranchesE2E(t *testing.T) {
if args["coverPermission"] != "true" || len(args["roleSubResourceIds"].([]string)) != 2 {
t.Fatalf("share args=%#v", args)
}
continueFailure := &minutesE2ECaller{responses: map[string][]string{"minutes/remove_member_permission": {`{"result":{}}`, `{"success":true,"result":{}}`}}}
continueFailure := &minutesE2ECaller{responses: map[string][]string{
"minutes/get_minutes_basic_info": {`{"success":true,"result":{"taskUuid":"u1"}}`},
"minutes/remove_member_permission": {
`{"result":{}}`,
`{"success":true,"result":{"resultMap":{"u1":["m2"]}}}`,
},
}}
payload, output, err := runMinutesAlignmentCLI(t, continueFailure, "minutes", "+unshare", "--id", "u1", "--member-uids", "m1,m2", "--failure-policy", "continue", "--yes")
if err == nil || output == "" || payload["failed"] != float64(1) || payload["succeeded"] != float64(1) {
t.Fatalf("continue ledger payload=%#v err=%v", payload, err)
}
missing := &minutesE2ECaller{responses: map[string][]string{
"minutes/get_minutes_basic_info": {`{"success":true,"result":{}}`},
}}
payload, output, err = runMinutesAlignmentCLI(t, missing, "minutes", "+unshare", "--id", "missing", "--member-uids", "m1", "--yes")
if err == nil || payload != nil || output != "" || missing.counts["minutes/remove_member_permission"] != 0 {
t.Fatalf("missing minutes reached unshare: payload=%#v output=%q err=%v calls=%#v", payload, output, err, missing.counts)
}
preflightFailure := &minutesE2ECaller{failAt: map[string]int{"minutes/get_minutes_basic_info": 1}}
payload, output, err = runMinutesAlignmentCLI(t, preflightFailure, "minutes", "+unshare", "--id", "unavailable", "--member-uids", "m1", "--yes")
if err == nil || payload != nil || output != "" || preflightFailure.counts["minutes/remove_member_permission"] != 0 {
t.Fatalf("failed preflight reached unshare: payload=%#v output=%q err=%v calls=%#v", payload, output, err, preflightFailure.counts)
}
}
func TestCrossPlatformCoverageMinutesArtifactCollectorBranches(t *testing.T) {
@@ -419,10 +442,9 @@ func TestCrossPlatformCoverageMinutesArtifactWaitAndOutput(t *testing.T) {
t.Fatalf("cancel failures=%#v", failures)
}
cmd = &cobra.Command{Use: "wait"}
cmd.SetContext(context.Background())
rt = shortcut.RuntimeContextForTest(cmd, ExportPack)
if err := waitMinutesInterval(rt, 0); err != nil {
t.Fatalf("timer wait: %v", err)
t.Fatalf("timer wait with unset command context: %v", err)
}
cmd.SetOut(minutesFailWriter{})
@@ -564,7 +586,7 @@ func TestCrossPlatformCoverageMinutesExportPackBranchesE2E(t *testing.T) {
}
func TestCrossPlatformCoverageMinutesExportPathAndJSONFaults(t *testing.T) {
if _, _, _, err := prepareExportTarget("../escape"); err == nil {
if _, _, _, err := prepareExportTarget(filepath.Join("..", "escape")); err == nil {
t.Fatal("unsafe target accepted")
}
for _, test := range []struct {
@@ -590,7 +612,7 @@ func TestCrossPlatformCoverageMinutesExportPathAndJSONFaults(t *testing.T) {
}
}},
{name: "escape", run: func(t *testing.T) {
testseam.Swap(t, &minutesRel, func(string, string) (string, error) { return "../escape", nil })
testseam.Swap(t, &minutesRel, func(string, string) (string, error) { return filepath.Join("..", "escape"), nil })
if _, _, _, err := prepareExportTarget("pack"); err == nil {
t.Fatal("rel escape accepted")
}
@@ -602,7 +624,7 @@ func TestCrossPlatformCoverageMinutesExportPathAndJSONFaults(t *testing.T) {
}
}},
{name: "parent", run: func(t *testing.T) {
testseam.Swap(t, &minutesRel, func(string, string) (string, error) { return "parent/pack", nil })
testseam.Swap(t, &minutesRel, func(string, string) (string, error) { return filepath.Join("parent", "pack"), nil })
testseam.Swap(t, &minutesLstat, func(string) (os.FileInfo, error) { return nil, errors.New("parent") })
if _, _, _, err := prepareExportTarget("pack"); err == nil {
t.Fatal("unsafe parent accepted")
@@ -224,6 +224,33 @@ func TestCrossPlatformCoverageMinutesWorkflowCompletion(t *testing.T) {
if err := RequireWriteAcknowledgement("write", map[string]any{"result": map[string]any{"updated": true}}); err != nil {
t.Fatal(err)
}
if err := RequirePermissionMutationAcknowledgement("unshare", []string{"u1"}, []string{"202397"}, map[string]any{
"success": true,
"result": map[string]any{"resultMap": map[string]any{"u1": []any{float64(202397)}}},
}); err != nil {
t.Fatalf("valid permission acknowledgement rejected: %v", err)
}
for _, data := range []map[string]any{
{"success": false, "errorMsg": "denied"},
{"success": "yes"},
{"success": true, "result": "bad"},
{"success": true, "result": map[string]any{}},
{"success": true, "result": map[string]any{"resultMap": map[string]any{}}},
{"success": true, "result": map[string]any{"resultMap": map[string]any{"other": []any{"202397"}}}},
{"success": true, "result": map[string]any{"resultMap": map[string]any{"u1": "bad"}}},
{"success": true, "result": map[string]any{"resultMap": map[string]any{"u1": []any{nil}}}},
{"success": true, "result": map[string]any{"resultMap": map[string]any{"u1": []any{"other"}}}},
} {
if err := RequirePermissionMutationAcknowledgement("unshare", []string{"u1"}, []string{"202397"}, data); err == nil {
t.Fatalf("invalid permission acknowledgement accepted: %#v", data)
}
}
if err := RequirePermissionMutationAcknowledgement("unshare", []string{"u1"}, []string{"202397", "other"}, map[string]any{
"success": true,
"result": map[string]any{"resultMap": map[string]any{"u1": []any{"202397"}}},
}); err == nil {
t.Fatal("incomplete member coverage accepted")
}
for _, tc := range []struct {
cmd, id string
data map[string]any
+55
View File
@@ -30,6 +30,61 @@ func RequireWriteAcknowledgement(operation string, data map[string]any) error {
return fmt.Errorf("minutes %s response has no explicit successful acknowledgement", operation)
}
// RequirePermissionMutationAcknowledgement validates the target-level payload
// returned by add/remove_member_permission. The backend's top-level
// success=true is not sufficient: historically it also accompanied a
// resultMap that merely echoed nonexistent task UUIDs.
func RequirePermissionMutationAcknowledgement(operation string, taskUUIDs, memberUIDs []string, data map[string]any) error {
if err := validateEnvelope(data); err != nil {
return err
}
if success, ok := data["success"].(bool); !ok || !success {
return fmt.Errorf("minutes %s response has no explicit success=true", operation)
}
result, ok := data["result"].(map[string]any)
if !ok {
return fmt.Errorf("minutes %s response has no result object", operation)
}
resultMap, ok := result["resultMap"].(map[string]any)
if !ok {
return fmt.Errorf("minutes %s response has no result.resultMap object", operation)
}
if len(resultMap) != len(taskUUIDs) {
return fmt.Errorf("minutes %s resultMap covers %d task UUIDs, want %d", operation, len(resultMap), len(taskUUIDs))
}
wantMembers := make(map[string]bool, len(memberUIDs))
for _, member := range memberUIDs {
wantMembers[strings.TrimSpace(member)] = true
}
for _, taskUUID := range taskUUIDs {
rawMembers, exists := resultMap[taskUUID]
if !exists {
return fmt.Errorf("minutes %s resultMap is missing task UUID %s", operation, taskUUID)
}
members, ok := rawMembers.([]any)
if !ok {
return fmt.Errorf("minutes %s resultMap[%s] has type %T, want array", operation, taskUUID, rawMembers)
}
gotMembers := make(map[string]bool, len(members))
for index, member := range members {
value := stringField(map[string]any{"member": member}, "member")
if value == "" {
return fmt.Errorf("minutes %s resultMap[%s][%d] is not a member UID", operation, taskUUID, index)
}
gotMembers[value] = true
}
if len(gotMembers) != len(wantMembers) {
return fmt.Errorf("minutes %s resultMap[%s] member coverage mismatch", operation, taskUUID)
}
for member := range wantMembers {
if !gotMembers[member] {
return fmt.Errorf("minutes %s resultMap[%s] is missing member UID %s", operation, taskUUID, member)
}
}
}
return nil
}
// RecordResult validates the gateway's observed listening-note command result.
func RecordResult(expectedCmd, taskUUID string, data map[string]any) (map[string]any, error) {
if err := RequireWriteAcknowledgement("record "+expectedCmd, data); err != nil {
+6 -13
View File
@@ -57,7 +57,7 @@ var ChatMessages = shortcut.Shortcut{
Command: "+chat-messages",
Product: "chat",
Description: "读取指定群聊或单聊的消息记录,支持有界全量分页与原子 JSON 导出",
Intent: "当你要读取或导出一个指定群聊或单聊的消息记录时使用;--sender 是可选过滤条件,可传姓名、userId 或 openDingTalkId:姓名优先唯一解析,稳定 ID 精确路由;通讯录无法分类时仍按原值 userId 筛选,但无稳定 senderId 命中不得作完整否定结论。不传时原样读取会话且不查询发送者身份。sender 仅是展示名,身份只比较稳定 senderId;" +
Intent: "当你要读取或导出一个指定群聊或单聊的消息记录时使用;--sender 是可选的姓名、userId 或 openDingTalkId 混合入口:姓名优先唯一解析,稳定 ID 精确路由;通讯录无法分类时仍按原值 userId 筛选并保留 identity_unverified,可交付精确命中但不能把原值升级为已验证身份或作完整否定结论。--sender-query 只按姓名唯一解析,解析失败会抑制未过滤消息并返回错误。不传发送者条件时原样读取会话且不查询发送者身份。sender 展示名不参与身份比较;" +
"群聊的 --group 可传群名或 openConversationId,单聊可传 --user 或 --open-dingtalk-id,所有目标参数互斥且必须选一个。自然群名只在唯一解析后读取,多候选会返回结构化 candidates。" +
"省略时间参数时默认从当前时间向前读取最近消息;兼容模式可用 --time/--direction,范围模式可用公开可选的 --start/--end/--order(兼容 --start-time/--end-time/--sort),范围语义为 [start,end)。" +
"全量读取用 --page-all,并由 --page-limit/--max-items 保持有界;结果公开 complete、hasMore、nextPage、stopReason、截断和逐页失败,不能把部分结果称为完整。--output 把同一 ledger 原子写为工作目录内 JSON。" +
@@ -83,7 +83,7 @@ var ChatMessages = shortcut.Shortcut{
},
Selection: contract.SelectionSpec{
AgentSummary: "读取指定群聊或单聊的消息记录,支持有界全量分页与原子 JSON 导出",
UseWhen: []string{"当你要读取或导出一个指定群聊或单聊的消息记录时使用;--sender 是可选过滤条件,可传姓名、userId 或 openDingTalkId:姓名优先唯一解析,稳定 ID 精确路由;通讯录无法分类时仍按原值 userId 筛选,但无稳定 senderId 命中不得作完整否定结论。不传时原样读取会话且不查询发送者身份。sender 仅是展示名,身份只比较稳定 senderId;" +
UseWhen: []string{"当你要读取或导出一个指定群聊或单聊的消息记录时使用;--sender 是可选的姓名、userId 或 openDingTalkId 混合入口:姓名优先唯一解析,稳定 ID 精确路由;通讯录无法分类时仍按原值 userId 筛选并保留 identity_unverified,可交付精确命中但不能把原值升级为已验证身份或作完整否定结论。--sender-query 只按姓名唯一解析,解析失败会抑制未过滤消息并返回错误。不传发送者条件时原样读取会话且不查询发送者身份。sender 展示名不参与身份比较;" +
"群聊的 --group 可传群名或 openConversationId,单聊可传 --user 或 --open-dingtalk-id,所有目标参数互斥且必须选一个。自然群名只在唯一解析后读取,多候选会返回结构化 candidates。" +
"省略时间参数时默认从当前时间向前读取最近消息;兼容模式可用 --time/--direction,范围模式可用公开可选的 --start/--end/--order(兼容 --start-time/--end-time/--sort),范围语义为 [start,end)。" +
"全量读取用 --page-all,并由 --page-limit/--max-items 保持有界;结果公开 complete、hasMore、nextPage、stopReason、截断和逐页失败,不能把部分结果称为完整。--output 把同一 ledger 原子写为工作目录内 JSON。" +
@@ -104,8 +104,8 @@ var ChatMessages = shortcut.Shortcut{
{Name: "user", Type: shortcut.FlagString, Desc: "单聊对方的 userId,与 --group 互斥"},
{Name: "user-query", Type: shortcut.FlagString, Desc: "按姓名解析唯一 openDingTalkId 的兼容入口", Hidden: true},
{Name: "open-dingtalk-id", Type: shortcut.FlagString, Desc: "单聊对方的 openDingTalkId,与 --group/--user 互斥"},
{Name: "sender", Type: shortcut.FlagStringSlice, Desc: "单个或多个发送者姓名、userId 或 openDingTalkId;姓名唯一解析,稳定 ID 精确路由,通讯录故障不阻断原值 userId 筛选"},
{Name: "sender-query", Type: shortcut.FlagStringSlice, Desc: "显式按姓名唯一解析发送者的兼容入口(可选,可重复或逗号分隔)"},
{Name: "sender", Type: shortcut.FlagStringSlice, Desc: "单个或多个发送者姓名、userId 或 openDingTalkId;姓名唯一解析,稳定 ID 精确路由,通讯录无法分类时按原值 userId 筛选并保留身份未验证状态"},
{Name: "sender-query", Type: shortcut.FlagStringSlice, Desc: "显式按姓名唯一解析发送者的兼容入口;解析失败时抑制未过滤消息并返回错误(可选,可重复或逗号分隔)"},
{Name: "time", Type: shortcut.FlagString, Desc: "时间边界,如 \"2025-03-01 00:00:00\";--time 必须是 RFC3339、YYYY-MM-DD HH:mm:ss 或 YYYY-MM-DD;省略时从当前时间向前读取最近消息"},
{Name: "start", Type: shortcut.FlagString, Desc: "范围开始时间(可选、包含),支持 RFC3339、YYYY-MM-DD HH:mm:ss 或 YYYY-MM-DD"},
{Name: "start-time", Type: shortcut.FlagString, Desc: "--start 的 lark-cli 对齐别名(可选、包含)"},
@@ -418,7 +418,7 @@ func applyOptionalChatMessagesSenderFilter(
payload["complete"] = false
payload["partial"] = len(filtered) > 0
}
unverifiedSenderInputs := chatMessagesUnverifiedSenderInputs(filtered, filter.resolutions)
unverifiedSenderInputs := chatMessagesUnverifiedSenderInputs(filter.resolutions)
if len(unverifiedSenderInputs) > 0 {
failures, _ := payload["failures"].([]map[string]any)
failures = append(failures, map[string]any{
@@ -470,18 +470,11 @@ func applyOptionalChatMessagesSenderFilter(
}
func chatMessagesUnverifiedSenderInputs(
messages []map[string]any,
resolutions []targetresolver.UserResolution,
) []string {
observed := map[string]bool{}
for _, message := range messages {
if senderID := strings.TrimSpace(fmt.Sprint(chatmsg.SenderID(message))); senderID != "" && senderID != "<nil>" {
observed[senderID] = true
}
}
values := make([]string, 0)
for _, resolution := range resolutions {
if targetresolver.IsUnverifiedUserIDResolution(resolution) && !observed[resolution.Selected.UserID] {
if targetresolver.IsUnverifiedUserIDResolution(resolution) {
values = append(values, resolution.Query)
}
}
@@ -631,7 +631,7 @@ func TestCrossPlatformCoverageChatMessagesSenderFilterFailureEdges(t *testing.T)
}
})
t.Run("unverified mixed sender becomes verified by an exact sender id match", func(t *testing.T) {
t.Run("unverified mixed sender remains unverified after an exact sender id match", func(t *testing.T) {
payload := map[string]any{"complete": true, "failures": []map[string]any{}}
resolution := targetresolver.UserResolution{
Status: targetresolver.StatusResolved,
@@ -655,13 +655,18 @@ func TestCrossPlatformCoverageChatMessagesSenderFilterFailureEdges(t *testing.T)
{"openMessageId": "other", "senderUserId": "other-user"},
}, &filter,
)
if len(filtered) != 1 || payload["complete"] != true || len(payload["failures"].([]map[string]any)) != 0 {
if len(filtered) != 1 || payload["complete"] != false || payload["partial"] != true ||
payload["failedCount"] != 1 || len(payload["failures"].([]map[string]any)) != 1 {
t.Fatalf("filtered=%#v payload=%#v", filtered, payload)
}
identity := payload["identityResult"].(map[string]any)
if identity["status"] != "evaluated" || identity["negativeConclusionAllowed"] != true {
if identity["status"] != "identity_unverified" || identity["negativeConclusionAllowed"] != false {
t.Fatalf("identityResult=%#v", identity)
}
filterResult := payload["senderFilter"].(map[string]any)
if filterResult["status"] != "identity_unverified" {
t.Fatalf("senderFilter=%#v", filterResult)
}
})
t.Run("unverified mixed sender without a match blocks complete negative conclusion", func(t *testing.T) {
+4 -11
View File
@@ -35,7 +35,7 @@ import (
// search hits through list_messages_by_ids in chunks of 50. A later-page or
// enrichment failure never turns a partial result into a false success: the
// output carries an explicit failure ledger and complete=false.
const searchMsgIntent = "当你要按关键词、发送者、@对象、消息类型、机器人来源或会话范围组合搜索 IM 消息时使用;可搜索单个、多个或全部会话。--group/--groups 接受群名或 openConversationId,--sender/--senders 接受姓名、userId 或 openDingTalkId:姓名优先唯一解析,稳定 ID 精确路由;通讯录无法分类时仍按原值 userId 执行,但无稳定 senderId 命中不得作完整否定结论。默认查询近 7 天,也可指定精确起止时间和输出顺序。" +
const searchMsgIntent = "当你要按关键词、发送者、@对象、消息类型、机器人来源或会话范围组合搜索 IM 消息时使用;可搜索单个、多个或全部会话。--group/--groups 接受群名或 openConversationId,--sender/--senders 接受姓名、userId 或 openDingTalkId:姓名优先唯一解析,稳定 ID 精确路由;通讯录无法分类时仍按原值 userId 执行并保留 identity_unverified,可交付精确命中但不能把原值升级为已验证身份或作完整否定结论。默认查询近 7 天,也可指定精确起止时间和输出顺序。" +
"显式指定会话时会先验证 CID,再执行有界全局扫描并在本地精确过滤,避免下层忽略非法 CID 或群聊 CID。" +
"--page-all 会连续拉取游标页,默认再按消息 ID 分批富化详情;任何续页或富化失败都会保留已取得结果并返回逐项失败 ledger,绝不把截断结果标成完整。" +
"--download-resources 使用安全本地路径、默认不覆盖和原子落盘。"
@@ -86,7 +86,7 @@ var SearchMsg = shortcut.Shortcut{
{Name: "groups", Type: shortcut.FlagStringSlice, Desc: "多个群名或 openConversationId;可混合输入并逐项唯一解析"},
{Name: "chat-id", Type: shortcut.FlagStringSlice, Desc: "--groups 的 lark-cli 对齐别名;只接受 openConversationId"},
{Name: "chat-query", Type: shortcut.FlagStringSlice, Desc: "显式按群名唯一解析的兼容入口(可选,可重复或逗号分隔)"},
{Name: "senders", Type: shortcut.FlagStringSlice, Desc: "多个发送者姓名、userId 或 openDingTalkId;姓名唯一解析,稳定 ID 精确路由,通讯录故障不阻断原值 userId 查询"},
{Name: "senders", Type: shortcut.FlagStringSlice, Desc: "多个发送者姓名、userId 或 openDingTalkId;姓名唯一解析,稳定 ID 精确路由,通讯录无法分类时按原值 userId 查询并保留身份未验证状态"},
{Name: "sender", Type: shortcut.FlagStringSlice, Desc: "单个或多个发送者姓名、userId 或 openDingTalkId;--senders 的兼容别名,保留同样的三态解析与安全降级语义"},
{Name: "sender-query", Type: shortcut.FlagStringSlice, Desc: "显式按姓名唯一解析的兼容入口(可选,可重复或逗号分隔)"},
{Name: "at-me", Type: shortcut.FlagBool, Desc: "只搜索 @我 的消息"},
@@ -278,7 +278,7 @@ var SearchMsg = shortcut.Shortcut{
return searchSenderScopeUnverifiedError(resolvedFilters.Senders, unverifiableMessageIDs)
}
}
unverifiedSenderInputs := searchUnverifiedSenderInputs(messages, resolvedFilters.Senders)
unverifiedSenderInputs := searchUnverifiedSenderInputs(resolvedFilters.Senders)
if len(unverifiedSenderInputs) > 0 {
failures = append(failures, map[string]any{
"stage": "sender_identity_verification",
@@ -701,18 +701,11 @@ func filterSearchSenderScope(
}
func searchUnverifiedSenderInputs(
messages []map[string]any,
resolutions []targetresolver.UserResolution,
) []string {
observed := map[string]bool{}
for _, message := range messages {
if senderID := strings.TrimSpace(fmt.Sprint(chatmsg.SenderID(message))); senderID != "" && senderID != "<nil>" {
observed[senderID] = true
}
}
values := make([]string, 0)
for _, resolution := range resolutions {
if targetresolver.IsUnverifiedUserIDResolution(resolution) && !observed[resolution.Selected.UserID] {
if targetresolver.IsUnverifiedUserIDResolution(resolution) {
values = append(values, resolution.Query)
}
}
@@ -277,9 +277,14 @@ func TestCrossPlatformCoverageSearchMsgStableUserIDContinuesWhenDirectoryIsUnava
if _, exists := caller.calls[1].args["senderOpenDingTakIds"]; exists {
t.Fatalf("unrelated candidate leaked into request: %#v", caller.calls[1].args)
}
if payload["complete"] != true || payload["count"] != float64(1) {
if payload["complete"] != false || payload["count"] != float64(1) ||
payload["failedCount"] != float64(1) {
t.Fatalf("payload=%#v", payload)
}
scope := payload["senderScope"].(map[string]any)
if scope["status"] != "identity_unverified" || scope["targetsResolved"] != false {
t.Fatalf("senderScope=%#v", scope)
}
})
t.Run("mixed sender preserves positive matches and blocks complete negative conclusion", func(t *testing.T) {
@@ -298,11 +303,11 @@ func TestCrossPlatformCoverageSearchMsgStableUserIDContinuesWhenDirectoryIsUnava
if got := caller.calls[1].args["senderUserIds"]; !reflect.DeepEqual(got, []string{"stable-user-id"}) {
t.Fatalf("senderUserIds=%#v", got)
}
if payload["count"] != float64(1) || payload["complete"] != true || payload["failedCount"] != float64(0) {
if payload["count"] != float64(1) || payload["complete"] != false || payload["failedCount"] != float64(1) {
t.Fatalf("payload=%#v", payload)
}
scope := payload["senderScope"].(map[string]any)
if _, exists := scope["status"]; exists || scope["targetsResolved"] != true {
if scope["status"] != "identity_unverified" || scope["targetsResolved"] != false {
t.Fatalf("senderScope=%#v", scope)
}
})
+115 -21
View File
@@ -6,16 +6,35 @@ ROOT="$(CDPATH= cd -- "$(dirname -- "$0")/../.." && pwd)"
usage() {
printf '%s\n' \
"usage: $0 verify <app-package>" \
" $0 run <app-package>" >&2
" $0 run <app-package> [partition]" \
" $0 list-partitions" >&2
exit 2
}
[ "$#" -eq 2 ] || usage
mode="$1"
app_package="$2"
# Single source of truth for the partition set. CI runs one job per partition and
# pins its shard names to this list, so a name that appears here without a
# dispatch entry below fails closed rather than silently skipping tests.
APP_PARTITIONS='schema a-b c d-r s-z-example-fuzz'
mode="${1:-}"
partition=""
case "$mode" in
verify|run) ;;
list-partitions)
[ "$#" -eq 1 ] || usage
for name in $APP_PARTITIONS; do
printf '%s\n' "$name"
done
exit 0
;;
verify)
[ "$#" -eq 2 ] || usage
app_package="$2"
;;
run)
[ "$#" -eq 2 ] || [ "$#" -eq 3 ] || usage
app_package="$2"
partition="${3:-}"
;;
*) usage ;;
esac
@@ -70,21 +89,49 @@ if [ "$unmatched_count" -ne 0 ]; then
exit 1
fi
for partition in \
# The loop variable is deliberately not named "partition": that name holds the
# partition requested on the command line, and run mode still executes this
# discovery pass before dispatching.
classified=''
for spec in \
"schema:$schema_count" \
"a-b:$ab_count" \
"c:$c_count" \
"d-r:$dr_count" \
"s-z-example-fuzz:$sz_count"
do
name="${partition%%:*}"
count="${partition#*:}"
name="${spec%%:*}"
count="${spec#*:}"
classified="$classified $name"
if [ "$count" -eq 0 ]; then
printf 'app race partition %s is empty\n' "$name" >&2
exit 1
fi
done
# The counters above and APP_PARTITIONS must describe the same set in both
# directions. A counted partition missing from APP_PARTITIONS would never be
# dispatched by any CI job, and a dispatchable partition with no counter would
# escape the exact-coverage check above.
for name in $APP_PARTITIONS; do
case " $classified " in
*" $name "*) ;;
*)
printf 'app partition %s has no coverage counter\n' "$name" >&2
exit 1
;;
esac
done
for name in $classified; do
case " $APP_PARTITIONS " in
*" $name "*) ;;
*)
printf 'counted partition %s is not a dispatchable app partition\n' "$name" >&2
exit 1
;;
esac
done
total_count="$(wc -l < "$tests" | tr -d ' ')"
assigned_count=$((schema_count + ab_count + c_count + dr_count + sz_count))
if [ "$assigned_count" -ne "$total_count" ]; then
@@ -101,26 +148,73 @@ fi
run_partition() {
name="$1"
run_pattern="$2"
skip_pattern="${3:-}"
instrumentation="$2"
run_pattern="$3"
skip_pattern="${4:-}"
printf 'running internal/app race partition %s\n' "$name"
# Fail closed on an unrecognized mode: a typo must not silently drop race
# instrumentation from a partition that is supposed to carry it.
case "$instrumentation" in
race|no-race) ;;
*)
printf 'unknown instrumentation %s for app partition %s\n' \
"$instrumentation" "$name" >&2
exit 1
;;
esac
printf 'running internal/app %s partition %s\n' "$instrumentation" "$name"
set -- -v -count=1 -timeout=15m -run "$run_pattern"
if [ -n "$skip_pattern" ]; then
go test -v -race -count=1 -timeout=15m \
-run "$run_pattern" -skip "$skip_pattern" "$app_package"
else
go test -v -race -count=1 -timeout=15m \
-run "$run_pattern" "$app_package"
set -- "$@" -skip "$skip_pattern"
fi
if [ "$instrumentation" = race ]; then
set -- -race "$@"
fi
go test "$@" "$app_package"
}
# Schema assembly has the largest transient memory footprint. Run those tests
# in a fresh process, then keep each remaining name range in its own process so
# command trees retained by process-global registries are released between
# partitions. The complementary run/skip patterns preserve the full test set.
#
# The schema partition runs uninstrumented. Its tests assert structural
# Schema-to-Cobra contracts over a single goroutine: none of them call
# t.Parallel or start a goroutine, so the race detector has no concurrent access
# to observe here. The process-global lazy metadata that does need race coverage
# (schema_source_root's atomic.Value, the parameter-binding lazy loaders) is
# exercised by internal/cli's concurrent tests, which stay instrumented. The
# instrumentation is not free on this partition: its shared sync.Once Catalog
# build is allocation-heavy, and -race made the partition roughly 11x slower
# (26s -> 291s locally, 357s in CI) without being able to report anything.
schema_pattern='^Test.*Schema'
run_partition schema "$schema_pattern"
run_partition a-b '^Test[A-B]' "$schema_pattern"
run_partition c '^TestC' "$schema_pattern"
run_partition d-r '^Test[D-R]' "$schema_pattern"
run_partition s-z-example-fuzz '^(Test[S-Z]|Example|Fuzz)' "$schema_pattern"
# Dispatch table for the partition set declared in APP_PARTITIONS. CI passes one
# partition per job so they run concurrently; running without a partition keeps
# the original end-to-end behaviour for local use and for any caller that wants
# the whole package in one invocation.
run_named_partition() {
case "$1" in
schema) run_partition schema no-race "$schema_pattern" ;;
a-b) run_partition a-b race '^Test[A-B]' "$schema_pattern" ;;
c) run_partition c race '^TestC' "$schema_pattern" ;;
d-r) run_partition d-r race '^Test[D-R]' "$schema_pattern" ;;
s-z-example-fuzz)
run_partition s-z-example-fuzz race '^(Test[S-Z]|Example|Fuzz)' "$schema_pattern"
;;
*)
printf 'unknown app partition: %s\n' "$1" >&2
exit 1
;;
esac
}
if [ -n "$partition" ]; then
run_named_partition "$partition"
exit 0
fi
for name in $APP_PARTITIONS; do
run_named_partition "$name"
done
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,286 @@
#!/usr/bin/env python3
"""Fast unit tests for eval_tool_search_ranking.py."""
from __future__ import annotations
import importlib.util
import json
import math
import sys
import tempfile
import unittest
from pathlib import Path
SCRIPT = Path(__file__).with_name("eval_tool_search_ranking.py")
SPEC = importlib.util.spec_from_file_location("eval_tool_search_ranking", SCRIPT)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"cannot import {SCRIPT}")
ranking = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = ranking
SPEC.loader.exec_module(ranking)
class ToolSearchRankingEvalTest(unittest.TestCase):
def tool(self, canonical: str, summary: str) -> ranking.ToolDocument:
return ranking.ToolDocument(
canonical=canonical,
product=canonical.partition(".")[0],
cli_path=canonical.replace(".", " "),
identities=(canonical,),
fields={
"identity": canonical,
"summary": summary,
"description": "",
"parameters": "",
},
use_when=(),
avoid_when=(),
examples=(),
)
def test_tokenize_preserves_identifier_and_splits_mixed_text(self) -> None:
tokens = ranking.tokenize("chat.send_personal_message 给群聊发送FilePath")
self.assertIn("chat.send_personal_message", tokens)
self.assertIn("群聊", tokens)
self.assertIn("file", tokens)
self.assertIn("path", tokens)
def test_parameter_projection_separates_names_descriptions_and_types(self) -> None:
parameters = {
"file": {
"property": "filePath",
"description": "本地文件路径",
"interface_description": "上传的媒体文件",
"type": "string",
"interface_type": "path",
"enum": ["document"],
}
}
names = ranking.parameter_text(
parameters,
include_names=True,
include_descriptions=False,
include_types=False,
)
self.assertIn("filePath", names)
self.assertIn("document", names)
self.assertNotIn("本地文件路径", names)
descriptions = ranking.parameter_text(
parameters,
include_names=False,
include_descriptions=True,
include_types=False,
)
self.assertIn("本地文件路径", descriptions)
self.assertIn("上传的媒体文件", descriptions)
self.assertNotIn("filePath", descriptions)
types = ranking.parameter_text(
parameters,
include_names=False,
include_descriptions=False,
include_types=True,
)
self.assertEqual(types, "string path")
def test_load_catalog_merges_build_time_product_shards(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
catalog_dir = Path(temporary)
tools_dir = catalog_dir / "tools"
tools_dir.mkdir()
(catalog_dir / "catalog.json").write_text(
json.dumps(
{
"source_hash": "sha256:source",
"surface_hash": "sha256:surface",
"catalog": {
"agent_metadata": {"surface_tools": 2}
},
}
),
encoding="utf-8",
)
pairs = [("chat", "chat.send"), ("drive", "drive.get")]
for product, canonical in pairs:
(tools_dir / f"{product}.json").write_text(
json.dumps(
{
"product": product,
"tools": {
canonical: {
"canonical_path": canonical,
"cli_path": canonical.replace(".", " "),
"agent_summary": canonical,
"product_id": product,
}
},
}
),
encoding="utf-8",
)
source_hash, tools = ranking.load_catalog(catalog_dir)
self.assertEqual(source_hash, "sha256:source")
self.assertEqual(
[tool.canonical for tool in tools], ["chat.send", "drive.get"]
)
def test_load_catalog_rejects_duplicate_shard_tools(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
catalog_dir = Path(temporary)
tools_dir = catalog_dir / "tools"
tools_dir.mkdir()
(catalog_dir / "catalog.json").write_text(
json.dumps({"catalog": {"agent_metadata": {"surface_tools": 1}}}),
encoding="utf-8",
)
shard = {"tools": {"chat.send": {"canonical_path": "chat.send"}}}
for name in ["a.json", "b.json"]:
(tools_dir / name).write_text(json.dumps(shard), encoding="utf-8")
with self.assertRaisesRegex(ValueError, "duplicate tools"):
ranking.load_catalog(catalog_dir)
def test_bm25_ranks_relevant_tool_first(self) -> None:
tools = [
self.tool("chat.send", "发送群聊消息"),
self.tool("drive.download", "下载钉盘文件"),
]
for ranker_type in [
ranking.BM25Ranker,
ranking.BM25LRanker,
ranking.BM25PlusRanker,
]:
with self.subTest(ranker=ranker_type.__name__):
ranker = ranker_type(tools)
self.assertEqual(
ranker.rank_many(["群聊发消息"])["群聊发消息"][0],
"chat.send",
)
def test_vector_and_set_rankers_rank_relevant_chinese_first(self) -> None:
tools = [
self.tool("chat.send", "发送群聊消息"),
self.tool("drive.download", "下载钉盘文件"),
]
for ranker_type in [ranking.TfidfCosineRanker, ranking.WeightedJaccardRanker]:
with self.subTest(ranker=ranker_type.__name__):
ranker = ranker_type(tools)
self.assertEqual(ranker.rank("群聊发消息")[0], "chat.send")
def test_stable_rank_breaks_ties_by_canonical(self) -> None:
self.assertEqual(
ranking.stable_rank({"z.tool": 1.0, "a.tool": 1.0}),
["a.tool", "z.tool"],
)
def test_rrf_rewards_agreement(self) -> None:
fused = ranking.rrf_rank(
[["solo", "shared"], ["other", "shared"]]
)
self.assertEqual(fused[0], "shared")
def test_rrf_k_changes_tail_influence_but_not_tie_break(self) -> None:
rankings = [["a", "shared"], ["b", "shared"]]
self.assertEqual(ranking.rrf_rank(rankings, k=10.0)[0], "shared")
self.assertEqual(ranking.rrf_rank(rankings, k=60.0)[0], "shared")
def test_forbidden_metric_is_lower_when_forbidden_tool_is_absent(self) -> None:
cases = [ranking.SingleCase("query", "danger.delete", "danger")]
metrics = ranking.evaluate_forbidden(cases, {"query": ["safe.read"]})
self.assertEqual(metrics["forbidden_at_5"], 0.0)
def test_workflow_decomposition_round_robins_steps(self) -> None:
case = ranking.WorkflowCase(
case_id="sample",
query="do both",
required=("first", "second"),
subqueries=("step one", "step two"),
)
result = ranking.decompose_workflow_rankings(
[case],
{
"step one": ["first", "shared"],
"step two": ["second", "shared"],
},
)
self.assertEqual(result["do both"][:2], ["first", "second"])
def test_workflow_ndcg_rewards_required_tools_near_the_top(self) -> None:
case = ranking.WorkflowCase(
case_id="sample",
query="do both",
required=("first", "second"),
subqueries=(),
)
best = ranking.evaluate_workflows(
[case], {"do both": ["first", "second", "other"]}
)
delayed = ranking.evaluate_workflows(
[case], {"do both": ["other", "first", "second"]}
)
self.assertEqual(best["mean_ndcg_at_5"], 1.0)
self.assertGreater(
best["mean_ndcg_at_5"], delayed["mean_ndcg_at_5"]
)
def test_query_language_slices(self) -> None:
self.assertEqual(ranking.query_language_slice("给群里发文件"), "chinese_only")
self.assertEqual(
ranking.query_language_slice("读取 baseId 对应表格"),
"mixed_chinese_ascii",
)
self.assertEqual(ranking.query_language_slice("chat send"), "non_chinese")
def test_bm25_variant_formulas_match_hand_calculation(self) -> None:
documents = {"doc": ["term", "term"], "other": ["other"]}
length_ratio = 2.0 / 1.5
idf = math.log(2.0)
query_weight = math.log(2.0)
plus = ranking.BM25VariantIndex(documents, variant="bm25plus")
plus_contribution = 2.0 * 1.9 / (
2.0 + 0.9 * (0.6 + 0.4 * length_ratio)
) + 1.0
self.assertAlmostEqual(
plus.scores("term")["doc"],
idf * plus_contribution * query_weight,
)
bm25l = ranking.BM25VariantIndex(documents, variant="bm25l")
normalized = 2.0 / (0.6 + 0.4 * length_ratio)
bm25l_contribution = 1.9 * (normalized + 1.0) / (
0.9 + normalized + 1.0
)
self.assertAlmostEqual(
bm25l.scores("term")["doc"],
idf * bm25l_contribution * query_weight,
)
def test_product_cluster_bootstrap_uses_equal_product_weight(self) -> None:
cases = [
ranking.SingleCase("a1", "a", "large"),
ranking.SingleCase("a2", "a", "large"),
ranking.SingleCase("a3", "a", "large"),
ranking.SingleCase("b1", "b", "small"),
]
result = ranking.paired_product_cluster_bootstrap_interval(
cases,
baseline=[False, False, False, True],
candidate=[True, True, True, False],
samples=200,
)
self.assertEqual(result["product_clusters"], 2)
self.assertAlmostEqual(result["macro_product_recall_at_5_delta"], 0.0)
def test_percentile_uses_nearest_rank(self) -> None:
values = [5.0, 1.0, 4.0, 2.0, 3.0]
self.assertEqual(ranking.percentile(values, 0.5), 3.0)
self.assertEqual(ranking.percentile(values, 0.95), 5.0)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,313 @@
#!/usr/bin/env python3
"""Verify frozen Tool Search evaluation inputs and independent qrels gates."""
from __future__ import annotations
import argparse
import json
import re
import subprocess
from pathlib import Path
from typing import Any
REPO_ROOT = Path(__file__).resolve().parents[2]
DEFAULT_MANIFEST = REPO_ROOT / "scripts/testdata/tool_search_eval_manifest.json"
DEFAULT_CATALOG_DIR = REPO_ROOT / ".worktrees/policy-tmp/tool-search-schema-catalog"
CHINESE = re.compile(r"[\u3400-\u9fff]")
ASCII_WORD = re.compile(r"[A-Za-z0-9]")
def language_slice(query: str) -> str:
has_chinese = bool(CHINESE.search(query))
has_ascii = bool(ASCII_WORD.search(query))
if has_chinese and has_ascii:
return "mixed_chinese_ascii"
if has_chinese:
return "chinese_only"
return "english"
def require_string(value: Any, field: str, problems: list[str]) -> str:
if not isinstance(value, str) or not value.strip():
problems.append(f"{field} must be a non-empty string")
return ""
return value.strip()
def validate_case(case: Any, index: int, catalog_tools: set[str], problems: list[str]) -> tuple[str, bool]:
prefix = f"cases[{index}]"
if not isinstance(case, dict):
problems.append(f"{prefix} must be an object")
return "", False
require_string(case.get("id"), f"{prefix}.id", problems)
query = require_string(case.get("query"), f"{prefix}.query", problems)
declared_language = require_string(case.get("language"), f"{prefix}.language", problems)
actual_language = language_slice(query) if query else ""
if declared_language and declared_language != actual_language:
problems.append(
f"{prefix}.language={declared_language!r} does not match {actual_language!r}"
)
qrels = case.get("qrels")
if not isinstance(qrels, list) or not qrels:
problems.append(f"{prefix}.qrels must be a non-empty list")
else:
seen: set[str] = set()
for qrel_index, qrel in enumerate(qrels):
qrel_prefix = f"{prefix}.qrels[{qrel_index}]"
if not isinstance(qrel, dict):
problems.append(f"{qrel_prefix} must be an object")
continue
canonical = require_string(qrel.get("canonical"), f"{qrel_prefix}.canonical", problems)
relevance = qrel.get("relevance")
if canonical in seen:
problems.append(f"{qrel_prefix}.canonical duplicates {canonical!r}")
seen.add(canonical)
if canonical and canonical not in catalog_tools:
problems.append(f"{qrel_prefix}.canonical {canonical!r} is not in the Catalog")
if not isinstance(relevance, int) or relevance not in {1, 2, 3}:
problems.append(f"{qrel_prefix}.relevance must be 1, 2 or 3")
forbidden = case.get("forbidden", [])
alternatives = case.get("alternative_gold", [])
if forbidden:
valid_forbidden = isinstance(forbidden, list) and all(isinstance(item, str) and item.strip() for item in forbidden)
if not valid_forbidden:
problems.append(f"{prefix}.forbidden must be a non-empty string list")
elif len(set(forbidden)) != len(forbidden):
problems.append(f"{prefix}.forbidden must not contain duplicates")
elif any(item not in catalog_tools for item in forbidden):
problems.append(f"{prefix}.forbidden contains an unknown Catalog tool")
valid_alternatives = isinstance(alternatives, list) and bool(alternatives) and all(
isinstance(item, str) and item.strip() for item in alternatives
)
if not valid_alternatives:
problems.append(f"{prefix}.alternative_gold is required when forbidden is non-empty")
elif len(set(alternatives)) != len(alternatives):
problems.append(f"{prefix}.alternative_gold must not contain duplicates")
elif any(item not in catalog_tools for item in alternatives):
problems.append(f"{prefix}.alternative_gold contains an unknown Catalog tool")
if valid_forbidden and valid_alternatives and set(forbidden).intersection(alternatives):
problems.append(f"{prefix}.forbidden and alternative_gold must be disjoint")
elif alternatives:
problems.append(f"{prefix}.alternative_gold requires a non-empty forbidden list")
workflow = case.get("workflow")
is_workflow = workflow is not None
if is_workflow:
if not isinstance(workflow, dict):
problems.append(f"{prefix}.workflow must be an object")
else:
required = workflow.get("required", [])
if not isinstance(required, list) or not 2 <= len(required) <= 4:
problems.append(f"{prefix}.workflow.required must contain 2 to 4 tools")
elif any(item not in catalog_tools for item in required):
problems.append(f"{prefix}.workflow.required contains an unknown Catalog tool")
subqueries = workflow.get("subqueries", [])
if subqueries and (
not isinstance(subqueries, list)
or not all(isinstance(item, str) and item.strip() for item in subqueries)
):
problems.append(f"{prefix}.workflow.subqueries must be a non-empty string list when present")
return actual_language, is_workflow
def load_generated_catalog(catalog_dir: Path, problems: list[str]) -> tuple[dict[str, Any], set[str], dict[str, dict[str, Any]]]:
envelope_path = catalog_dir / "catalog.json"
tools_dir = catalog_dir / "tools"
if not envelope_path.is_file():
problems.append(f"generated Catalog envelope does not exist: {envelope_path}")
return {}, set(), {}
if not tools_dir.is_dir():
problems.append(f"generated Catalog tools directory does not exist: {tools_dir}")
return {}, set(), {}
envelope = json.loads(envelope_path.read_text(encoding="utf-8"))
tools: set[str] = set()
metadata: dict[str, dict[str, Any]] = {}
for shard_path in sorted(tools_dir.glob("*.json")):
shard = json.loads(shard_path.read_text(encoding="utf-8"))
shard_tools = shard.get("tools")
if not isinstance(shard_tools, dict):
problems.append(f"generated Catalog shard has no tools object: {shard_path}")
continue
overlap = tools.intersection(shard_tools)
if overlap:
problems.append(f"generated Catalog shards duplicate tools: {sorted(overlap)}")
tools.update(shard_tools)
metadata.update(shard_tools)
return envelope, tools, metadata
def validate_manifest(
manifest_path: Path,
catalog_dir: Path | None = None,
) -> list[str]:
problems: list[str] = []
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
if manifest.get("version") != "tool-search-eval-manifest.v1":
problems.append("manifest version must be tool-search-eval-manifest.v1")
base_commit = require_string(manifest.get("base_repository_commit"), "base_repository_commit", problems)
if base_commit:
exists = subprocess.run(
["git", "cat-file", "-e", f"{base_commit}^{{commit}}"],
cwd=REPO_ROOT,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if exists.returncode != 0:
problems.append("base_repository_commit does not exist")
else:
ancestor = subprocess.run(
["git", "merge-base", "--is-ancestor", base_commit, "HEAD"],
cwd=REPO_ROOT,
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
if ancestor.returncode != 0:
problems.append("base_repository_commit is not an ancestor of HEAD")
owners = manifest.get("owners", {})
retrieval_owner = require_string(owners.get("retrieval"), "owners.retrieval", problems)
evaluation_owner = require_string(
owners.get("independent_evaluation"), "owners.independent_evaluation", problems
)
if retrieval_owner and retrieval_owner == evaluation_owner:
problems.append("retrieval and independent evaluation owners must differ")
require_string(owners.get("separation_rule"), "owners.separation_rule", problems)
proxy = manifest.get("proxy_v1", {})
# Source files evolve with every ranking change; the manifest deliberately
# does not pin their content hashes (that would couple every code change
# to a manifest update). Only their existence is asserted. Behavior is
# gated by the Go evaluation sentinels instead.
for path_field in ["workflow_path", "evaluator_path"]:
relative = require_string(proxy.get(path_field), f"proxy_v1.{path_field}", problems)
if relative and not (REPO_ROOT / relative).is_file():
problems.append(f"{relative} does not exist")
catalog_payload: dict[str, Any] = {}
catalog_tools: set[str] = set()
catalog_metadata: dict[str, dict[str, Any]] = {}
if catalog_dir is not None:
catalog_payload, catalog_tools, catalog_metadata = load_generated_catalog(catalog_dir, problems)
if len(catalog_tools) != proxy.get("tool_count"):
problems.append("proxy_v1.tool_count differs from generated Catalog")
if catalog_payload.get("source_hash") != proxy.get("catalog_source_hash"):
problems.append("proxy_v1.catalog_source_hash differs from generated Catalog")
if catalog_payload.get("surface_hash") != proxy.get("catalog_surface_hash"):
problems.append("proxy_v1.catalog_surface_hash differs from generated Catalog")
independent = manifest.get("independent_test_v1", {})
qrels_relative = require_string(independent.get("path"), "independent_test_v1.path", problems)
qrels_path = REPO_ROOT / qrels_relative
if not qrels_path.is_file():
problems.append(f"{qrels_relative} does not exist")
return problems
qrels = json.loads(qrels_path.read_text(encoding="utf-8"))
if qrels.get("version") != "tool-search-qrels.v1":
problems.append("independent qrels version must be tool-search-qrels.v1")
cases = qrels.get("cases")
if not isinstance(cases, list):
problems.append("independent qrels cases must be a list")
cases = []
ids: set[str] = set()
for index, case in enumerate(cases):
if isinstance(case, dict) and isinstance(case.get("id"), str):
if case["id"] in ids:
problems.append(f"cases[{index}].id duplicates {case['id']!r}")
ids.add(case["id"])
validate_case(case, index, catalog_tools, problems)
# Coverage derivation also validates qrels data quality (confusion_family
# consistency) as a side effect; the release-time sealed gate that consumed
# the coverage set has been removed.
derive_independent_coverage(cases, catalog_metadata, problems)
return problems
def derive_independent_coverage(
cases: list[Any], catalog_metadata: dict[str, dict[str, Any]], problems: list[str]
) -> set[str]:
"""Derive release coverage from qrels and Catalog facts, never self-asserted labels."""
coverage: set[str] = set()
if not catalog_metadata:
return coverage
catalog_products = {str(tool.get("product_id", "")) for tool in catalog_metadata.values()}
judged: set[str] = set()
effects: set[str] = set()
workflow_ok = False
graded_equivalent = False
forbidden_alternative = False
sibling_confusion = False
for index, case in enumerate(cases):
if not isinstance(case, dict):
continue
canonicals = [
qrel.get("canonical") for qrel in case.get("qrels", []) if isinstance(qrel, dict)
]
for canonical in canonicals:
if canonical in catalog_metadata:
judged.add(canonical)
effects.add(str(catalog_metadata[canonical].get("effect", "")))
if len(canonicals) >= 2:
backing = {
json.dumps(catalog_metadata.get(item, {}).get("interface_ref", {}), sort_keys=True)
for item in canonicals
}
relevances = {
qrel.get("relevance") for qrel in case.get("qrels", []) if isinstance(qrel, dict)
}
if len(backing) == 1 and "{}" not in backing and len(relevances) >= 2:
graded_equivalent = True
forbidden = case.get("forbidden", [])
alternatives = case.get("alternative_gold", [])
if forbidden and alternatives:
forbidden_alternative = True
workflow = case.get("workflow")
if isinstance(workflow, dict) and 2 <= len(workflow.get("required", [])) <= 4:
workflow_ok = True
confusion = case.get("confusion_family", [])
if confusion:
if not isinstance(confusion, list) or len(confusion) < 2 or any(item not in catalog_metadata for item in confusion):
problems.append(f"cases[{index}].confusion_family must contain at least two Catalog tools")
elif set(confusion).intersection(canonicals):
problems.append(f"cases[{index}].confusion_family must be disjoint from qrels")
elif len({catalog_metadata[item].get("product_id") for item in confusion}) != 1:
problems.append(f"cases[{index}].confusion_family must belong to one product")
else:
sibling_confusion = True
judged_products = {str(catalog_metadata[item].get("product_id", "")) for item in judged}
if judged_products == catalog_products:
coverage.add("all_reviewed_products")
if {"read", "write", "destructive"}.issubset(effects):
coverage.add("read_write_destructive")
if sibling_confusion:
coverage.add("sibling_confusion")
if graded_equivalent:
coverage.add("graded_equivalent_entrypoints")
if forbidden_alternative:
coverage.add("forbidden_with_alternative_gold")
if workflow_ok:
coverage.add("two_to_four_step_workflows")
return coverage
def main() -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--manifest", type=Path, default=DEFAULT_MANIFEST)
parser.add_argument("--catalog-dir", type=Path, default=DEFAULT_CATALOG_DIR)
args = parser.parse_args()
problems = validate_manifest(
args.manifest.resolve(),
args.catalog_dir.resolve(),
)
if problems:
for problem in problems:
print(f"ERROR: {problem}")
return 1
print("tool search evaluation manifest: OK")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Unit tests for verify_tool_search_eval_manifest.py."""
from __future__ import annotations
import importlib.util
import sys
import unittest
from pathlib import Path
SCRIPT = Path(__file__).with_name("verify_tool_search_eval_manifest.py")
SPEC = importlib.util.spec_from_file_location("verify_tool_search_eval_manifest", SCRIPT)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"cannot import {SCRIPT}")
verify = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = verify
SPEC.loader.exec_module(verify)
class ToolSearchEvalManifestTest(unittest.TestCase):
def test_language_slice(self) -> None:
self.assertEqual(verify.language_slice("给群里发文件"), "chinese_only")
self.assertEqual(
verify.language_slice("读取 baseId 对应的表格"),
"mixed_chinese_ascii",
)
self.assertEqual(verify.language_slice("send a file to the group"), "english")
def test_collecting_manifest_is_valid(self) -> None:
self.assertEqual(verify.validate_manifest(verify.DEFAULT_MANIFEST), [])
def test_graded_equivalent_qrels_are_accepted(self) -> None:
catalog = {
"dev.search_open_platform_docs_rag",
"devdoc.search_open_platform_docs_rag",
}
case = {
"id": "devdoc-search",
"query": "查 OAuth2 接入文档",
"language": "mixed_chinese_ascii",
"qrels": [
{
"canonical": "devdoc.search_open_platform_docs_rag",
"relevance": 3,
},
{
"canonical": "dev.search_open_platform_docs_rag",
"relevance": 2,
},
],
}
problems: list[str] = []
language, workflow = verify.validate_case(case, 0, catalog, problems)
self.assertEqual(language, "mixed_chinese_ascii")
self.assertFalse(workflow)
self.assertEqual(problems, [])
def test_forbidden_requires_alternative_gold(self) -> None:
case = {
"id": "unsafe",
"query": "删除记录",
"language": "chinese_only",
"qrels": [{"canonical": "safe.read", "relevance": 3}],
"forbidden": ["danger.delete"],
}
problems: list[str] = []
verify.validate_case(case, 0, {"safe.read", "danger.delete"}, problems)
self.assertTrue(any("alternative_gold" in problem for problem in problems))
def test_forbidden_and_alternative_gold_are_catalog_bound_and_disjoint(self) -> None:
case = {
"id": "unsafe",
"query": "删除记录",
"language": "chinese_only",
"qrels": [{"canonical": "safe.read", "relevance": 3}],
"forbidden": ["danger.delete", "unknown.tool"],
"alternative_gold": ["danger.delete"],
}
problems: list[str] = []
verify.validate_case(case, 0, {"safe.read", "danger.delete"}, problems)
self.assertTrue(any("unknown Catalog tool" in problem for problem in problems), problems)
self.assertTrue(any("must be disjoint" in problem for problem in problems), problems)
def test_release_coverage_is_derived_not_self_asserted(self) -> None:
metadata = {
"chat.send": {"product_id": "chat", "effect": "write", "interface_ref": {"rpc_name": "send"}},
"chat.delete": {"product_id": "chat", "effect": "destructive", "interface_ref": {"rpc_name": "delete"}},
"doc.read": {"product_id": "doc", "effect": "read", "interface_ref": {"rpc_name": "get"}},
"doc.fetch": {"product_id": "devdoc", "effect": "read", "interface_ref": {"rpc_name": "get"}},
}
cases = [
{
"qrels": [{"canonical": "chat.send", "relevance": 3}],
"forbidden": ["chat.delete"],
"alternative_gold": ["chat.send"],
"confusion_family": ["chat.delete", "chat.send.other"],
"workflow": {"required": ["chat.send", "doc.read"]},
},
{
"qrels": [
{"canonical": "doc.read", "relevance": 3},
{"canonical": "doc.fetch", "relevance": 2},
]
},
{"qrels": [{"canonical": "chat.delete", "relevance": 3}]},
]
problems: list[str] = []
coverage = verify.derive_independent_coverage(cases, metadata, problems)
self.assertIn("all_reviewed_products", coverage)
self.assertIn("read_write_destructive", coverage)
self.assertIn("graded_equivalent_entrypoints", coverage)
self.assertNotIn("sibling_confusion", coverage)
self.assertTrue(any("confusion_family" in problem for problem in problems))
if __name__ == "__main__":
unittest.main()
+3 -3
View File
@@ -136,7 +136,7 @@ def mono_overview(items: list[dict[str, Any]]) -> str:
return f"""{MONO_START}
## Shortcut 总览
下面只统计当前公开 catalog 中的 shortcut,不展开完整明细。已知意图应先按产品 Skill、意图表或任务 reference 选择唯一命令;命令已选中时直接执行,只在参数或安全语义不确定时读取 leaf Schema,在当前 Cobra flags 不确定时读取 leaf Help。仅当现有路由和 reference 都无法定位低频能力时,才用 `dws shortcut list --service <service> --format json` 做最后回退;不要为已知高频意图加载完整产品 Catalog。
下面只统计当前公开 catalog 中的 shortcut,不展开完整明细。已知意图应先按产品 Skill、意图表或任务 reference 选择唯一命令;命令已选中时直接执行,只在参数或安全语义不确定时读取 leaf Schema,在当前 Cobra flags 不确定时读取 leaf Help。仅当现有路由和 reference 都无法定位低频能力时,才用 `dws schema search --query "<用户意图>" --limit 5` 搜索原子命令和 shortcut;`dws shortcut list` 只作人工审计/兼容 fallback,不要为已知高频意图加载完整产品 Catalog。
| 服务 | shortcut 数 | multi skill |
|---|---:|---|
@@ -157,7 +157,7 @@ def product_section(service: str, rows: list[dict[str, Any]]) -> str:
return f"""{PRODUCT_START}
## Shortcuts(无专用脚本/recipe 时优先)
以下 shortcut 同时进入公开 catalog 与 Runtime Schema。先按本 skill 的意图表、脚本和 recipe 路由:存在精确覆盖该场景的专用脚本/recipe 时按其执行;否则用户意图命中时,shortcut 优先于手写原子命令。命令已选中时直接执行;只在参数或安全语义不确定时读取 Agent leaf Schema(例如 `dws schema --cli-path "{service} +<shortcut>" --compact --format json`),在当前 Cobra flags 不确定时读取 `dws {service} <shortcut> --help`。只有参数映射、接口绑定或 provenance 审计才省略 `--compact`。仅当现有路由和 reference 都无法定位低频能力时,才用 `dws shortcut list --service {service} --format json` 批量发现。
以下 shortcut 同时进入公开 catalog 与 Runtime Schema。先按本 skill 的意图表、脚本和 recipe 路由:存在精确覆盖该场景的专用脚本/recipe 时按其执行;否则用户意图命中时,shortcut 优先于手写原子命令。命令已选中时直接执行;只在参数或安全语义不确定时读取 Agent leaf Schema(例如 `dws schema --cli-path "{service} +<shortcut>" --compact --format json`),在当前 Cobra flags 不确定时读取 `dws {service} <shortcut> --help`。只有参数映射、接口绑定或 provenance 审计才省略 `--compact`。仅当现有路由和 reference 都无法定位低频能力时,才用 `dws schema search --query "<用户意图>" --product {service} --limit 5` 同时搜索原子命令和 shortcut。`dws shortcut list --service {service}` 只作人工审计/兼容 fallback。
| Shortcut | 风险 | 适用场景 |
|---|---|---|
@@ -179,7 +179,7 @@ def compact_product_section(service: str, rows: list[dict[str, Any]]) -> str:
`{md_escape(service)}` 当前有 {public_count} 条公开 shortcut,完整清单保留在 Runtime Catalog 与 Schema,不在高频产品根 Skill 中重复展开。{discovery}
仅当现有路由和 reference 都无法定位低频能力时,才执行 `dws shortcut list --service {md_escape(service)} --format json` 做最后回退;不要为已知高频意图加载完整 Shortcut Catalog 或产品级 Schema。
仅当现有路由和 reference 都无法定位低频能力时,才执行 `dws schema search --query "<用户意图>" --product {md_escape(service)} --limit 5` 同时搜索原子命令和 shortcut。`dws shortcut list` 只作人工审计/兼容 fallback;不要为已知高频意图加载完整 Shortcut Catalog 或产品级 Schema。
{PRODUCT_END}"""
+1
View File
@@ -116,5 +116,6 @@ fi
# Assembly determinism validates fresh CI/local Catalog dumps.
"$ROOT/scripts/policy/check-schema-assembly.sh"
python3 "$ROOT/scripts/policy/check-shortcut-public-catalog-drift.py"
printf 'generated drift check: ok\n'

Some files were not shown because too many files have changed in this diff Show More