Self-review follow-ups on the ranking hardening series:
- the avoid_when boilerplate filter compared lowercased phrases against
a mixed-case constant, so the "需要该 Shortcut..." noise filter never
fired; the constant is now lowercase (no ranking change: the boilerplate
was never contained by any query, sentinel 990/1123 unchanged)
- add direct unit coverage for the avoid_when penalty (echoed-phrase
match, unrelated-query pass-through, boilerplate filter, partial
overlap, explain annotation)
- update tool-search-ranking-evaluation.md to the post-hardening
numbers (ensemble R@5 88.16%, Forbidden 0%/0%, action_v1 shadow
84.24% with the trade-off recorded)
- drop the dead algorithm_freeze manifest block; nothing consumes it
after the hash decoupling
- default engine: fielded BM25 ensemble (action_v1 remains a shadow candidate)
- add --explain score_breakdown: per-field contributions, structured
multiplier, query classification and avoid_when penalty
- add DWS_TOOL_SEARCH_* env overrides (algorithm, k1/b, field weights,
explain) for release-independent tuning and rollback
- tokenizer: CJK unigram+bigram so single-character intents match; align
isToolSearchCJK with unicode.Han used by the evaluation slices
- structured routing: extend vocabularies with English synonyms,
word-boundary matching for ASCII values, restrict the rerank gate to
technical identifiers only
- avoid_when demotion layer wraps every lexical algorithm (Forbidden@5
proxy 37.55% -> 0); intent-overlap matching rejected after control
showed it demoted gold answers (Recall@5 0.8664 -> 0.7845)
- remove the OA task_id +1.00 fixture-specific multiplier
- build the default engine once per process for long-lived hosts
- evaluation baseline: intent-proxy R@5 990/1123 (88.16%); update
integration sentinels accordingly
Address the two P1 review findings and the coverage-gate CI failures:
- Every install/upgrade path that removes a skill dir (opposite-mode
leftovers, stale dingtalk-* / dws-shared, and same-name refreshes) now
moves the directory to ~/.dws/skill-backups/<stamp>/ first across
install.sh, install-skills.sh, install.ps1, install.js, `dws skill
setup`, and `dws upgrade`. A backup failure preserves the original
directory and never removes it.
- Remove --yes from every copyable `dws skill setup` example and document
what the command may remove; add regression tests that declining the
confirmation performs no removal and that the confirmation previews
every directory slated for backup+removal.
- Rename the skill-mode tests to the TestCrossPlatformCoverage* prefix so
the platform coverage gate selects them, and add edge tests for the
backup/prune/cleanup fallback branches, restoring changed-code coverage
to 100%.
Co-authored-by: Cursor <cursoragent@cursor.com>
玉澜