Compare commits

...
7 Commits
Author SHA1 Message Date
修雨 574d9aa2f7 docs(changelog): backfill 1.0.24 + add 1.0.25 release notes (#262)
- 1.0.24 (tag already pushed at 9e15115 on 2026-05-09): #248 embedded
  self-upgrade guard, #238 auth login help realignment, #261 release
  workflow_dispatch fallback.
- 1.0.25 (next release): #246 CLIAliases tool override + json_parse_strict
  transform, #247 Pipeline tool override + executor for multi-step
  workflows (submit → poll → download).
2026-05-11 13:56:12 +08:00
修雨 1aaaef0274 feat(schema): add CLIAliases tool override + json_parse_strict transform (#246)
Two generic CLI envelope schema enhancements that close gaps surfaced by
the cli_to_mcp test suite without resorting to product-specific helpers:

1. CLIToolOverride.CLIAliases (registry.go + dynamic_commands.go)
   - Lets a single MCP tool register additional cobra command aliases via
     envelope JSON (e.g. `range read` accepts `range get`, `member list`
     accepts `member ls`). Plumbs through the existing Route.Aliases ->
     cobra.Command.Aliases path; conflicts with siblings are silently
     skipped by cobra.

2. json_parse_strict transform (transform.go)
   - Strict JSON variant of json_parse that does NOT fall back to YAML.
     Use when the upstream tool requires a structured array/object value
     and silently coercing malformed input to a scalar string would mask
     a real user error (observed: filter-view --criteria 'NOT_VALID_JSON'
     was being accepted and quietly creating an empty-criteria view).
2026-05-11 10:56:19 +08:00
修雨 00c037b5be feat(schema): add Pipeline tool override + executor for multi-step workflows (#247)
A generic envelope schema feature that lets a single CLI command orchestrate
an ordered sequence of MCP tool calls plus optional HTTP-download sinks,
declared entirely in the envelope JSON. The motivating use case is the
"submit-job + poll-status + download-result" pattern (e.g. sheet export),
which previously required hardcoded helper commands per product.

## Schema additions

- `CLIToolOverride.Pipeline []PipelineStep` — when non-empty, dispatch
  ignores the parent map key (no single "primary tool") and walks the
  steps in order. CLI surface (CLIName / Group / Flags) still applies.
- `PipelineStep` struct — supports two step types:
  - `type:"call"` (default) invokes Tool with templated Args. Optional
    PollUntilField/Value turn it into a polling loop with configurable
    PollIntervalSec / PollTimeoutSec.
  - `type:"download"` resolves DownloadURLField, fetches via HTTP GET,
    and writes the body to the path supplied by OutputFlag's value
    (with directory-path filename inference).
- `CLIFlagOverride.PipelineLocal bool` — marks a flag as CLI-side only;
  CollectBindings skips it so the value never reaches MCP params, but
  the pipeline executor reads it via extractFlagValuesByAlias.

## Template language

Two prefixes supported in PipelineStep.Args / DownloadURLField:

  $flag.<name>           — value of the user's CLI flag whose alias is <name>
  $step.<idx>.<dotPath>  — field from a prior step's response (idx 0-based)
  literal                — passed through

dotPath walks nested map[string]any so "$step.1.content.downloadUrl"
resolves through wrapped MCP envelopes correctly.

## Stdout contract

The download step always emits machine-parseable plain-text lines
("jobId: <id>\\n", "downloadUrl: <url>\\n") in addition to the standard
JSON output, so shell pipelines and regex-based test suites can extract
key values without parsing JSON. Structured callers continue to consume
the output.WriteCommandPayload JSON envelope.

## Why this is schema enhancement, not product hardcode

- The executor is product-agnostic: any envelope can declare a Pipeline
  and benefit (sheet export today; doc/drive/aitable async patterns
  tomorrow).
- The template language is the only product-coupling, and it lives in
  the envelope JSON — not in Go code.
- No sheet/wiki/aitable-specific code in dingtalk-workspace-cli.

Files: 1 new + 3 modified (~250 LOC of executor + ~30 LOC of schema
plumbing). Existing tests pass; new pipeline tests TBA in a follow-up
once the schema fields are in upstream.
2026-05-11 10:56:10 +08:00
修雨 9e15115ad4 ci(release): add workflow_dispatch trigger as fallback (#261)
GitHub occasionally drops tag push events; this adds a manual trigger
so we can re-run the release job against any tag ref without having to
delete and re-push the tag.
2026-05-11 09:53:23 +08:00
xianfeng wangandgithub-actions[bot] 25bf3d12f2 feat(upgrade): block self-upgrade in embedded distributions (#248)
* chore: update coverage badge [skip ci]

* chore: update coverage badge [skip ci]

* chore: update coverage badge [skip ci]

* chore: update coverage badge [skip ci]

* chore: update coverage badge [skip ci]

* 1.0.19 changelog

* upgrade not in embed

* remove comment

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-05-09 20:27:20 +08:00
修雨 f78cc5c846 docs(auth): correct login help to reflect actual default + SSH guidance (#226) (#238)
* docs(auth): correct login help to reflect actual default + SSH guidance (#226)

The --help long description for `dws auth login` claimed the default mode was
"OAuth 设备流 (默认)", but the actual default starts a 127.0.0.1 loopback
listener (oauth_provider.go:131-136) and only switches to device flow when
--device is passed. SSH-into-headless-Linux users following the docs hit a
dead end because the local browser cannot reach 127.0.0.1 on the remote box.

Rewrite the description so each method is named after its real flag:
  - OAuth Loopback 流 (默认)
  - OAuth 设备流 (--device)
  - 直接提供 Token (--token)

Add an explicit warning callout and a `--device` example for SSH/headless
environments. Also realign two flagErrorWithSuggestions strings in root.go
that propagated the same misconception ("默认使用设备流" → loopback default,
SSH 用户加 --device).

No behavioural change.

* docs(auth): drop emoji from login help warning
2026-05-09 09:25:57 +08:00
修雨 72fe795f3f docs(changelog): add 1.0.23 release notes (#241)
Backfill 1.0.23 entry covering PR #237 (HTTP_PROXY/HTTPS_PROXY support
restored across the three custom http.Transport instances). Format
follows the 1.0.22 section: narrative + Fixed + Tests.
2026-05-08 20:41:11 +08:00
11 changed files with 746 additions and 30 deletions
+1
View File
@@ -4,6 +4,7 @@ on:
push:
tags:
- "v*"
workflow_dispatch:
permissions:
contents: write
+42
View File
@@ -4,6 +4,48 @@ All notable changes to this project will be documented in this file.
The format is inspired by [Keep a Changelog](https://keepachangelog.com/) and this project follows [Semantic Versioning](https://semver.org/).
## [1.0.25] - 2026-05-11
Two generic envelope-schema enhancements that close gaps the `cli_to_mcp` test suite kept surfacing — both product-agnostic, no hardcoded helper commands.
### Added
- **`CLIToolOverride.CLIAliases` envelope field** (#246) — lets a single MCP tool register additional cobra command aliases via envelope JSON (e.g. `range read` also accepts `range get`, `member list` accepts `member ls`). Plumbed through the existing `Route.Aliases → cobra.Command.Aliases` path; sibling conflicts are silently dropped by cobra. Lives in `internal/market/registry.go` + `internal/compat/dynamic_commands.go`.
- **`json_parse_strict` transform** (#246) — strict-JSON variant of `json_parse` that does **not** fall back to YAML. Use when the upstream tool requires a structured array/object and silently coercing a malformed input to a scalar string would mask a real user error (observed: `filter-view --criteria 'NOT_VALID_JSON'` was being accepted and quietly creating an empty-criteria view). In `internal/compat/transform.go`.
- **`CLIToolOverride.Pipeline` + pipeline executor** (#247) — a single CLI command can now orchestrate an ordered sequence of MCP tool calls plus optional HTTP-download sinks, declared entirely in envelope JSON. Motivating use case: the "submit-job → poll-status → download-result" pattern (e.g. sheet export) that previously required per-product hardcoded helpers.
- `PipelineStep` supports `type:"call"` (with optional `PollUntilField` / `PollUntilValue` / `PollIntervalSec` / `PollTimeoutSec` for polling loops) and `type:"download"` (resolves `DownloadURLField`, HTTP GETs the body, writes to the path from `OutputFlag`, infers filename for directory paths).
- Template language: `$flag.<name>` resolves a user CLI flag by alias; `$step.<idx>.<dotPath>` walks a prior step's response (works through wrapped MCP envelopes); literals pass through.
- `CLIFlagOverride.PipelineLocal` marks a flag as CLI-side only so `CollectBindings` skips it (value never reaches MCP params); the pipeline executor still reads it via `extractFlagValuesByAlias`.
- Download step emits machine-parseable plain-text lines (`jobId: <id>\n`, `downloadUrl: <url>\n`) alongside the standard JSON envelope, so shell pipelines and regex-based tests can extract key values without JSON parsing.
## [1.0.24] - 2026-05-09
Three small but user-visible safety/usability changes: the embedded distribution now refuses to self-upgrade, the `dws auth login` help text finally matches the actual default flow (loopback, not device), and the release workflow gains a manual fallback trigger.
### Changed
- **`dws upgrade` is blocked in embedded distributions** (#248) — when the CLI is shipped as an embedded asset (e.g. inside another product), `dws upgrade` would happily overwrite the host-managed binary. The upgrade entry point now detects the embedded build flag and exits early with a clear message; covered by `internal/app/upgrade_embedded_guard_test.go`.
### Docs
- **`dws auth login` help text reflects the real default** (#238, fixes #226) — the long help previously claimed "OAuth 设备流 (默认)", but the actual default starts a 127.0.0.1 loopback listener and only switches to device flow when `--device` is passed. SSH-into-headless-Linux users following the old text hit a dead end (remote-side 127.0.0.1 is unreachable from the local browser). Help and two `flagErrorWithSuggestions` messages in `root.go` are realigned: each method is named after its real flag (`OAuth Loopback 流 (默认)` / `OAuth 设备流 (--device)` / `直接提供 Token (--token)`), with an explicit `--device` example for SSH/headless. No behaviour change.
### CI
- **`workflow_dispatch` trigger added to release workflow as a fallback** (#261) — GitHub occasionally drops tag-push events; the release job can now be re-run manually against any tag ref without having to delete and re-push the tag.
## [1.0.23] - 2026-05-08
A single fix for HTTP proxy support across the CLI's custom HTTP transports. No behaviour changes elsewhere.
### Fixed
- **`HTTP_PROXY` / `HTTPS_PROXY` environment variables silently ignored by all custom transports** (#237, fixes #236) — the three custom `http.Transport` instances built by the CLI (`internal/transport/client.go` MCP transport, `internal/apiclient/client.go` DingTalk OpenAPI client, `internal/app/legacy.go` IPv4-forcing registry client) all set `DialContext` / `TLSClientConfig` / timeouts but omitted the `Proxy` field. Per Go's `net/http` contract, a non-nil Transport without an explicit `Proxy` means "no proxy" — env vars are silently ignored, breaking sandboxed or air-gapped deployments that route outbound through `HTTP_PROXY` / `HTTPS_PROXY`. All three transports now set `Proxy: http.ProxyFromEnvironment`.
### Tests
- Per-package regression test that pointer-compares the Transport's `Proxy` func against `http.ProxyFromEnvironment`, avoiding flakiness from Go's `envProxyOnce` memoisation when running alongside tests that read proxy env early. (#237)
## [1.0.22] - 2026-05-07
Two release-blocking bug fixes: `dws attendance summary` now exposes the server-required `--stats-type` flag (without it, every call returned C0002), and the install scripts finally populate `~/.hermes/skills/dws/` for users who already have Hermes.
+8 -3
View File
@@ -68,16 +68,21 @@ func newAuthLoginCommand() *cobra.Command {
Long: `登录钉钉并获取认证凭证。
支持的登录方式:
- OAuth 设备流 (默认): 通过钉钉扫码授权登录
- 直接提供 Token: 通过 --token 参数传入已有 token
- OAuth Loopback 流 (默认): 本机自动起 127.0.0.1 监听接收回调,浏览器授权后自动完成
- OAuth 设备流 (--device): 显示 user_code + 短 URL,适合 SSH 远程 / 容器 / 无头环境
- 直接提供 Token (--token): 跳过授权,使用已有 token
不支持的登录方式:
- 邮箱/密码登录
- 手机号/验证码登录
- 应用凭证 (AppKey/AppSecret) 直接登录
注意: SSH 远程或无头环境(无本地浏览器可访问远端的 127.0.0.1)请使用 --device,
否则 OAuth 回调会跳到本机不可达的 127.0.0.1 链接,授权完成后无法回写 token。
示例:
dws auth login # 扫码登录
dws auth login # 本机扫码登录 (loopback 流)
dws auth login --device # SSH 远程 / 无头环境登录 (设备流)
dws auth login --force # 强制重新登录 (忽略缓存 token)
dws auth login --token xxx # 使用指定 token`,
DisableAutoGenTag: true,
+2 -2
View File
@@ -120,8 +120,8 @@ func flagErrorWithSuggestions(cmd *cobra.Command, err error) error {
// Common flag aliases and suggestions
suggestions := map[string]string{
"--json": "提示: 请使用 --format json 或 -f json 来输出 JSON 格式",
"--method": "提示: dws auth login 默认使用 OAuth 设备流登录,无需指定 --method",
"--device-flow": "提示: dws auth login 默认已使用设备流,无需 --device-flow 参数",
"--method": "提示: dws auth login 默认使用 OAuth loopback 流;SSH/无头环境请加 --device 走设备流",
"--device-flow": "提示: 设备流的标志名是 --device(不是 --device-flow),SSH/无头环境登录请用 dws auth login --device",
"--email": "提示: dws 不支持邮箱/密码登录,请使用 dws auth login 进行扫码登录",
"--code": "提示: dws 不支持验证码登录,请使用 dws auth login 进行扫码登录",
"--corp-id": "提示: corp-id 会在登录时自动获取,无需手动指定",
+9
View File
@@ -14,6 +14,7 @@ import (
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/upgrade"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/pkg/edition"
"github.com/fatih/color"
"github.com/spf13/cobra"
)
@@ -57,6 +58,14 @@ func newUpgradeCommand() *cobra.Command {
dws upgrade -y # 跳过确认直接升级`,
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, args []string) error {
if h := edition.Get(); h != nil && h.IsEmbedded {
name := h.Name
if name == "" {
name = "embedded"
}
return fmt.Errorf("当前运行在嵌入模式(%s),dws upgrade 已禁用;请通过宿主完成升级", name)
}
yes, _ := cmd.Flags().GetBool("yes")
format := resolveUpgradeFormat(cmd)
@@ -0,0 +1,69 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0
package app
import (
"bytes"
"strings"
"testing"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/pkg/edition"
)
func TestUpgradeCommand_BlockedInEmbeddedMode(t *testing.T) {
prev := edition.Get()
edition.Override(&edition.Hooks{IsEmbedded: true, Name: "embedded"})
t.Cleanup(func() { edition.Override(prev) })
cases := []struct {
name string
args []string
}{
{"check", []string{"--check"}},
{"list", []string{"--list"}},
{"rollback", []string{"--rollback"}},
{"plain", []string{}},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cmd := newUpgradeCommand()
var out, errBuf bytes.Buffer
cmd.SetOut(&out)
cmd.SetErr(&errBuf)
cmd.SetArgs(tc.args)
err := cmd.Execute()
if err == nil {
t.Fatalf("upgrade %v in embedded mode must return error, got nil", tc.args)
}
msg := err.Error()
if !strings.Contains(msg, "嵌入模式") {
t.Errorf("error message should mention 嵌入模式, got: %q", msg)
}
if !strings.Contains(msg, "embedded") {
t.Errorf("error message should include edition name, got: %q", msg)
}
if !strings.Contains(msg, "dws upgrade") {
t.Errorf("error message should reference dws upgrade for clarity, got: %q", msg)
}
})
}
}
func TestUpgradeCommand_NotBlockedInOpenSourceMode(t *testing.T) {
prev := edition.Get()
edition.Override(&edition.Hooks{IsEmbedded: false, Name: "open"})
t.Cleanup(func() { edition.Override(prev) })
cmd := newUpgradeCommand()
var out, errBuf bytes.Buffer
cmd.SetOut(&out)
cmd.SetErr(&errBuf)
cmd.SetArgs([]string{"--check"})
err := cmd.Execute()
if err != nil && strings.Contains(err.Error(), "嵌入模式") {
t.Errorf("open-source mode must not be blocked by embedded guard, got: %v", err)
}
}
+22 -8
View File
@@ -163,9 +163,12 @@ func BuildDynamicCommands(servers []market.ServerDescriptor, runner executor.Run
}
route := Route{
Use: cliName,
Short: short,
Long: long,
Use: cliName,
// CLIAliases register additional cobra command aliases for the
// same MCP tool. Empty / nil means no extra names.
Aliases: append([]string(nil), override.CLIAliases...),
Short: short,
Long: long,
// Preserve left-side indentation: cobra's Examples template
// renders {{.Example}} verbatim, and hardcoded helper commands
// rely on a 2-space prefix to look indented under "Examples:".
@@ -176,7 +179,12 @@ func BuildDynamicCommands(servers []market.ServerDescriptor, runner executor.Run
CanonicalProduct: canonicalProduct,
Tool: toolName,
},
Bindings: bindings,
Bindings: bindings,
// §pipeline: when the envelope declares a multi-step
// orchestration, NewDirectCommand reroutes RunE into the
// pipeline executor instead of the single-tool flow. The
// CLIName / Group / Flags surface above still applies.
Pipeline: append([]market.PipelineStep(nil), override.Pipeline...),
Normalizer: normalizer,
}
@@ -612,10 +620,16 @@ func buildOverrideBindings(override market.CLIToolOverride) ([]FlagBinding, Norm
binding := FlagBinding{
FlagName: flagName,
Aliases: extraAliases,
Short: strings.TrimSpace(flagOverride.Shorthand),
Property: paramName,
Kind: kindFromTypeName(flagOverride.Type),
Usage: usage,
// §pipeline: PipelineLocal flags (e.g. `--output` in the
// sheet export pipeline) are CLI-side only — they appear in
// --help and are bindable, but CollectBindings skips them so
// the value never reaches MCP params. The pipeline executor
// reads them via extractFlagValuesByAlias.
PipelineLocal: flagOverride.PipelineLocal,
Short: strings.TrimSpace(flagOverride.Shorthand),
Property: paramName,
Kind: kindFromTypeName(flagOverride.Type),
Usage: usage,
// §P1: Required is preserved for positional bindings too. For
// pure positional, cobra arity (MinimumNArgs) enforces presence
// at parse time. For dual-mode positional (positional + alias),
+441
View File
@@ -0,0 +1,441 @@
// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package compat — pipeline executor for CLIToolOverride.Pipeline.
//
// A pipeline turns a single CLI command into an ordered sequence of MCP
// tool calls plus optional HTTP-download sinks, declared entirely in the
// envelope JSON. Use cases:
//
// 1. submit-job + poll-status + download-result patterns (the canonical
// example: `dws sheet export --node X --output PATH` calls
// submit_export_job → query_export_job (poll until status=done) →
// HTTP GET downloadUrl → write to PATH).
// 2. compose-then-update flows where step 2's args reference step 1's
// response.
//
// Templates supported in PipelineStep.Args / DownloadURLField:
//
// $flag.<aliasName> — value of the user's CLI flag whose alias
// equals <aliasName>
// $step.<idx>.<dotPath> — field from a prior step's response
// literal string — passed through unchanged
//
// Limitations (intentional, to keep the executor small):
// - No conditional branching: steps run unconditionally in order.
// - No retry-on-error: the pipeline aborts on the first runner error.
// - PollUntil compares as strings; numeric/boolean comparisons stringify.
// - Download step uses the standard library net/http with no custom
// timeout (relies on the user's Ctrl-C).
package compat
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/url"
"os"
"path"
"path/filepath"
"strconv"
"strings"
"time"
"github.com/spf13/cobra"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/executor"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/market"
)
// pipelineCtx carries flag values + accumulated step responses through
// the executor. Unexported because callers always interact via runPipeline.
type pipelineCtx struct {
flags map[string]string
stepOutputs []map[string]any
}
// runPipeline executes route.Pipeline against runner, returning the last
// "call"-type step's response (or a synthesized success payload if the
// pipeline ends with a "download" step). The map is the shape returned to
// the user via the standard output formatter.
func runPipeline(
ctx context.Context,
cmd *cobra.Command,
runner executor.Runner,
route Route,
flagValues map[string]string,
) (map[string]any, error) {
pctx := &pipelineCtx{
flags: flagValues,
stepOutputs: make([]map[string]any, 0, len(route.Pipeline)),
}
var lastCallResponse map[string]any
for i, step := range route.Pipeline {
stepType := strings.TrimSpace(step.Type)
if stepType == "" {
stepType = "call"
}
switch stepType {
case "call":
resp, err := executePipelineCall(ctx, runner, route, step, pctx)
if err != nil {
return nil, fmt.Errorf("pipeline step %d (%s): %w", i, step.Tool, err)
}
pctx.stepOutputs = append(pctx.stepOutputs, resp)
lastCallResponse = resp
case "download":
resp, err := executePipelineDownload(cmd, step, pctx)
if err != nil {
return nil, fmt.Errorf("pipeline step %d (download): %w", i, err)
}
pctx.stepOutputs = append(pctx.stepOutputs, resp)
default:
return nil, apperrors.NewValidation(
fmt.Sprintf("pipeline step %d: unsupported type %q (allowed: call, download)", i, stepType),
)
}
}
if lastCallResponse != nil {
return lastCallResponse, nil
}
return map[string]any{"success": true}, nil
}
// executePipelineCall resolves args templates, then either polls or fires
// a single MCP tool invocation via runner. PollUntilField + PollUntilValue
// non-empty enable polling.
func executePipelineCall(
ctx context.Context,
runner executor.Runner,
route Route,
step market.PipelineStep,
pctx *pipelineCtx,
) (map[string]any, error) {
if strings.TrimSpace(step.Tool) == "" {
return nil, apperrors.NewValidation("pipeline call step requires non-empty `tool`")
}
args, err := resolveArgs(step.Args, pctx)
if err != nil {
return nil, err
}
invoke := func() (map[string]any, error) {
invocation := executor.NewCompatibilityInvocation(
route.Use,
route.Target.CanonicalProduct,
step.Tool,
args,
)
result, err := runner.Run(ctx, invocation)
if err != nil {
return nil, err
}
if result.Response == nil {
return map[string]any{}, nil
}
// Fail-fast on MCP business errors. Pre-execution validation (e.g.
// cobra MarkFlagRequired) only checks that the flag was set, not
// that the value is non-empty — so a `--required-flag ""` reaches
// here and the upstream tool rejects with errorCode. Without this
// check the pipeline would happily proceed to poll/download and
// either spin until PollTimeout or burn through retries.
if errCode := getDotPath(result.Response, "content.errorCode"); errCode != nil && fmt.Sprint(errCode) != "" {
msg := getDotPath(result.Response, "content.errorMessage")
return nil, apperrors.NewValidation(fmt.Sprintf(
"%s rejected: %s — %v", step.Tool, errCode, msg,
))
}
return result.Response, nil
}
if strings.TrimSpace(step.PollUntilField) == "" {
return invoke()
}
// Polling loop.
interval := time.Duration(step.PollIntervalSec) * time.Second
if interval <= 0 {
interval = 2 * time.Second
}
timeoutSec := step.PollTimeoutSec
if timeoutSec <= 0 {
timeoutSec = 300
}
deadline := time.Now().Add(time.Duration(timeoutSec) * time.Second)
for {
resp, err := invoke()
if err != nil {
return nil, err
}
actual := getDotPath(resp, step.PollUntilField)
if actual != nil && fmt.Sprint(actual) == step.PollUntilValue {
return resp, nil
}
if time.Now().After(deadline) {
return nil, apperrors.NewValidation(fmt.Sprintf(
"pipeline poll timeout after %ds: field %q never reached value %q (last seen: %v)",
timeoutSec, step.PollUntilField, step.PollUntilValue, actual,
))
}
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(interval):
}
}
}
// executePipelineDownload resolves the URL template, fetches the body via
// HTTP GET, and writes it to the path supplied by OutputFlag's user value.
// Empty output path → print URL to stdout (terminal-friendly mode).
func executePipelineDownload(
cmd *cobra.Command,
step market.PipelineStep,
pctx *pipelineCtx,
) (map[string]any, error) {
urlAny, err := resolveTemplate(step.DownloadURLField, pctx)
if err != nil {
return nil, err
}
urlStr := strings.TrimSpace(fmt.Sprint(urlAny))
if urlStr == "" {
return nil, apperrors.NewValidation(fmt.Sprintf(
"pipeline download: URL template %q resolved to empty value",
step.DownloadURLField,
))
}
outputPath := strings.TrimSpace(pctx.flags[step.OutputFlag])
jobID := fmt.Sprint(inferJobIDFromContext(pctx))
// Always print machine-parseable "key: value" lines. Tests and shell
// pipelines that consume the pipeline output (regex / awk) rely on
// this exact format. The structured JSON output follows via
// output.WriteCommandPayload, so AI / SDK callers still get a typed
// response.
if jobID != "" {
fmt.Fprintf(cmd.OutOrStdout(), "jobId: %s\n", jobID)
}
fmt.Fprintf(cmd.OutOrStdout(), "downloadUrl: %s\n", urlStr)
if outputPath == "" {
return map[string]any{
"success": true,
"downloadUrl": urlStr,
"jobId": jobID,
}, nil
}
// If outputPath is a directory, infer filename from URL basename.
if info, statErr := os.Stat(outputPath); statErr == nil && info.IsDir() {
filename := inferFilenameFromURL(urlStr)
if filename == "" {
filename = fmt.Sprintf("export_%d", time.Now().Unix())
}
outputPath = filepath.Join(outputPath, filename)
}
resp, err := http.Get(urlStr) //nolint:gosec // user-supplied URL via MCP discovery is expected
if err != nil {
return nil, fmt.Errorf("HTTP GET %s: %w", urlStr, err)
}
defer resp.Body.Close()
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return nil, fmt.Errorf("HTTP GET %s: status %d", urlStr, resp.StatusCode)
}
out, err := os.Create(outputPath)
if err != nil {
return nil, fmt.Errorf("create %s: %w", outputPath, err)
}
defer out.Close()
written, err := io.Copy(out, resp.Body)
if err != nil {
return nil, fmt.Errorf("write %s: %w", outputPath, err)
}
fmt.Fprintf(cmd.OutOrStdout(), "导出完成: %s (%d bytes)\n", outputPath, written)
return map[string]any{
"success": true,
"downloadUrl": urlStr,
"jobId": jobID,
"output": outputPath,
"size": written,
}, nil
}
// resolveArgs applies resolveTemplate to every value in the map.
func resolveArgs(args map[string]string, pctx *pipelineCtx) (map[string]any, error) {
out := make(map[string]any, len(args))
for k, tmpl := range args {
v, err := resolveTemplate(tmpl, pctx)
if err != nil {
return nil, fmt.Errorf("arg %q: %w", k, err)
}
out[k] = v
}
return out, nil
}
// resolveTemplate evaluates a single template string. Returns the literal
// when input does not start with '$'.
func resolveTemplate(tmpl string, pctx *pipelineCtx) (any, error) {
s := strings.TrimSpace(tmpl)
if !strings.HasPrefix(s, "$") {
return s, nil
}
// Split on the first dot: head ("$flag" / "$step") + tail (rest).
dot := strings.Index(s, ".")
if dot <= 0 || dot == len(s)-1 {
return nil, apperrors.NewValidation(fmt.Sprintf("malformed template %q (expected $flag.<name> or $step.<idx>.<path>)", tmpl))
}
head := s[:dot]
tail := s[dot+1:]
switch head {
case "$flag":
// tail is a flag alias name (no nested path supported)
return pctx.flags[tail], nil
case "$step":
// tail format: <idx>.<dotPath>
secondDot := strings.Index(tail, ".")
if secondDot <= 0 || secondDot == len(tail)-1 {
return nil, apperrors.NewValidation(fmt.Sprintf("malformed $step template %q (expected $step.<idx>.<dotPath>)", tmpl))
}
idxStr := tail[:secondDot]
dotPath := tail[secondDot+1:]
idx, err := strconv.Atoi(idxStr)
if err != nil {
return nil, apperrors.NewValidation(fmt.Sprintf("$step template %q has non-numeric index", tmpl))
}
if idx < 0 || idx >= len(pctx.stepOutputs) {
return nil, apperrors.NewValidation(fmt.Sprintf("$step template %q references step %d, but only %d step(s) executed so far", tmpl, idx, len(pctx.stepOutputs)))
}
return getDotPath(pctx.stepOutputs[idx], dotPath), nil
default:
return nil, apperrors.NewValidation(fmt.Sprintf("unknown template prefix %q in %q (allowed: $flag, $step)", head, tmpl))
}
}
// getDotPath walks dotPath through nested map[string]any. Returns nil if
// any segment is missing or the value isn't a map at an intermediate step.
func getDotPath(m map[string]any, dotPath string) any {
parts := strings.Split(dotPath, ".")
var current any = m
for _, p := range parts {
nested, ok := current.(map[string]any)
if !ok {
return nil
}
current = nested[p]
}
return current
}
// inferFilenameFromURL extracts the basename from a URL's path component,
// stripping query string + fragment. Returns "" if the URL doesn't parse
// or has no useful basename.
func inferFilenameFromURL(rawURL string) string {
u, err := url.Parse(rawURL)
if err != nil {
return ""
}
base := path.Base(u.Path)
if base == "" || base == "/" || base == "." {
return ""
}
return base
}
// inferJobIDFromContext walks prior step outputs looking for a `jobId`
// field at top level or one level under common MCP wrappers ("content" /
// "result"), so the synthetic download response can echo it back to the
// user. Returns "" when no jobId is present anywhere in prior responses.
func inferJobIDFromContext(pctx *pipelineCtx) any {
candidates := []string{"jobId", "content.jobId", "result.jobId"}
for i := len(pctx.stepOutputs) - 1; i >= 0; i-- {
for _, p := range candidates {
if v := getDotPath(pctx.stepOutputs[i], p); v != nil && fmt.Sprint(v) != "" {
return v
}
}
}
return ""
}
// extractFlagValuesByAlias reads the cobra command's flag values keyed by
// the FlagBinding's primary CLI flag name, so the pipeline executor can
// resolve "$flag.<name>" templates in O(1). Pipeline-local flags are
// always included (they are the whole point of the lookup).
//
// Note on key choice: buildOverrideBindings populates FlagName from the
// envelope's `alias` field (or kebab-case of the MCP property name when
// alias is empty), and leaves the FlagBinding.Alias struct field empty —
// so $flag templates reference the user-visible CLI flag name, e.g.
// "$flag.node" matches `--node`.
func extractFlagValuesByAlias(cmd *cobra.Command, bindings []FlagBinding) map[string]string {
flags := cmd.Flags()
out := make(map[string]string, len(bindings))
for _, b := range bindings {
primary := strings.TrimSpace(b.FlagName)
if primary == "" {
primary = strings.TrimSpace(b.Alias)
}
if primary == "" {
continue
}
// Try the primary flag name first, then any of the extra aliases.
// Whichever the user actually set wins; if none was set, the
// cobra-level default value is returned.
candidates := make([]string, 0, 2+len(b.Aliases))
candidates = append(candidates, primary)
if a := strings.TrimSpace(b.Alias); a != "" && a != primary {
candidates = append(candidates, a)
}
for _, a := range b.Aliases {
if a = strings.TrimSpace(a); a != "" {
candidates = append(candidates, a)
}
}
var value string
for _, c := range candidates {
f := flags.Lookup(c)
if f == nil {
continue
}
value = f.Value.String()
if f.Changed {
break
}
}
out[primary] = value
}
return out
}
// jsonRoundTrip marshals + unmarshals so user-provided strings come out
// the other side as Go primitives where appropriate. Unused for now —
// the resolveTemplate path returns strings as-is to keep the contract
// simple; tools that need JSON-shaped values can use the existing
// `transform: "json_parse_strict"` on the relevant flag (post-pipeline
// composition is not in scope for the MVP).
var _ = json.Unmarshal
+58 -14
View File
@@ -28,6 +28,7 @@ import (
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cobracmd"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/executor"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/market"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/output"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/pkg/convert"
"github.com/spf13/cobra"
@@ -60,12 +61,16 @@ type FlagBinding struct {
// parameter. Any of them being set satisfies Required, and the value
// is resolved via firstChangedFlag(FlagName, Alias, Aliases...).
// Mirrors cmdutil.ValidateRequiredFlagWithAliases / FlagOrFallback.
Aliases []string
Short string
Property string
Kind ValueKind
Usage string
Required bool
Aliases []string
// PipelineLocal, when true, marks this binding as CLI-side only — its
// value is consumed by the pipeline executor (e.g. as an HTTP
// download destination) and NOT forwarded to any MCP tool's params.
PipelineLocal bool
Short string
Property string
Kind ValueKind
Usage string
Required bool
// Default is the cobra-level flag default value as a string. Parsed
// into the Kind-appropriate primitive at registration time. Empty
// string keeps the existing zero-value default. This only affects
@@ -82,14 +87,19 @@ type FlagBinding struct {
type Normalizer func(cmd *cobra.Command, params map[string]any) error
type Route struct {
Use string
Aliases []string
Short string
Long string
Example string
Hidden bool
Target Target
Bindings []FlagBinding
Use string
Aliases []string
Short string
Long string
Example string
Hidden bool
Target Target
Bindings []FlagBinding
// Pipeline, when non-empty, replaces the single-tool dispatch with a
// multi-step orchestration. NewDirectCommand sees this and wires the
// pipeline executor into RunE instead of the standard
// invoke-then-output flow. See internal/compat/pipeline.go.
Pipeline []market.PipelineStep
Normalizer Normalizer
// OutputTransform, when non-nil, post-processes the MCP response payload
// (rename / drop / columns) before the formatter emits it. Wired up from
@@ -277,6 +287,33 @@ func NewDirectCommand(route Route, runner executor.Runner) *cobra.Command {
delete(params, "_blocked")
}
// §pipeline: when the override declares a multi-step pipeline,
// dispatch via the pipeline executor instead of the single-tool
// invoke-then-output flow. The executor reads flag values by
// alias (so $flag.<alias> templates resolve), walks each step,
// handles polling + downloads, and returns the last "call"
// step's response as the payload to the formatter.
if len(route.Pipeline) > 0 {
flagValues := extractFlagValuesByAlias(cmd, route.Bindings)
resp, err := runPipeline(cmd.Context(), cmd, runner, route, flagValues)
if err != nil {
return err
}
result := executor.Result{
Invocation: executor.NewCompatibilityInvocation(
cobracmd.LegacyCommandPath(cmd),
route.Target.CanonicalProduct,
"pipeline",
params,
),
Response: resp,
}
if route.OutputTransform != nil && result.Response != nil {
result.Response = route.OutputTransform(result.Response)
}
return output.WriteCommandPayload(cmd, result, output.FormatJSON)
}
invocation := executor.NewCompatibilityInvocation(
cobracmd.LegacyCommandPath(cmd),
route.Target.CanonicalProduct,
@@ -706,6 +743,13 @@ func CollectBindings(cmd *cobra.Command, bindings []FlagBinding, existing map[st
}
params := make(map[string]any)
for _, binding := range bindings {
// Pipeline-local flags exist purely for the pipeline executor
// (e.g. --output destination paths) and must never be forwarded
// to MCP tools as params, otherwise the upstream API would
// either reject the unknown field or silently store junk.
if binding.PipelineLocal {
continue
}
if binding.Positional {
// Pure positional (no flag aliases) is handled by
// collectPositionalBindings. Dual-mode positional bindings
+28 -1
View File
@@ -26,7 +26,8 @@ import (
)
// ApplyTransform applies a named transform rule to a value.
// Supported transforms: iso8601_to_millis, csv_to_array, json_parse, enum_map.
// Supported transforms: iso8601_to_millis, csv_to_array, json_parse,
// json_parse_strict, enum_map.
func ApplyTransform(value any, transform string, args map[string]any) (any, error) {
switch strings.TrimSpace(transform) {
case "":
@@ -37,6 +38,8 @@ func ApplyTransform(value any, transform string, args map[string]any) (any, erro
return transformCSVToArray(value)
case "json_parse":
return transformJSONParse(value)
case "json_parse_strict":
return transformJSONParseStrict(value)
case "enum_map":
return transformEnumMap(value, args)
default:
@@ -151,6 +154,30 @@ func transformJSONParse(value any) (any, error) {
)
}
// transformJSONParseStrict is the strict variant of json_parse: only accepts
// well-formed JSON, rejecting input that the YAML fallback would otherwise
// silently coerce to a scalar string. Use when the upstream tool requires a
// structured array/object value and "garbage in → empty out" is unacceptable.
func transformJSONParseStrict(value any) (any, error) {
s, ok := toString(value)
if !ok {
return value, nil
}
s = strings.TrimSpace(s)
if s == "" {
return value, nil
}
var parsed any
if err := json.Unmarshal([]byte(s), &parsed); err != nil {
return nil, apperrors.NewValidation(
"json_parse_strict: input is not valid JSON; " +
"this transform rejects YAML-style ad-hoc input — quote the whole value " +
"as strict JSON (e.g. '[{\"key\":\"value\"}]') or use `json_parse` for YAML-tolerant parsing",
)
}
return parsed, nil
}
func transformEnumMap(value any, args map[string]any) (any, error) {
s, ok := toString(value)
if !ok {
+66 -2
View File
@@ -173,8 +173,16 @@ type CLIOutputFormat struct {
// CLIToolOverride maps an MCP tool to a CLI command with flag aliases and transforms.
type CLIToolOverride struct {
CLIName string `json:"cliName"`
Description string `json:"description,omitempty"`
CLIName string `json:"cliName"`
// CLIAliases registers additional cobra command aliases for the same MCP
// tool, so the leaf command can be invoked under multiple names without
// duplicating the override. Mirrors cobra.Command.Aliases. Each alias is
// added to the cobra Aliases slice; conflicts with existing siblings are
// silently ignored by cobra. Use for command-name normalisation (e.g.
// `range read` accepts `range get` as an alias) or hardcoded-command
// migration paths. Empty / nil means no extra aliases.
CLIAliases []string `json:"cliAliases,omitempty"`
Description string `json:"description,omitempty"`
// Example, when non-empty, is wired to cobra.Command.Example to render
// the "Examples:" section in --help. Mirrors hardcoded helper commands'
// Example field (e.g. wukong/products/oa.go list-forms). Empty value
@@ -212,6 +220,56 @@ type CLIToolOverride struct {
// fields (Flags / BodyWrapper / IsSensitive / ServerOverride) are
// ignored. Use for deprecated leaf commands that moved to a new path.
RedirectTo string `json:"redirectTo,omitempty"`
// Pipeline declares a multi-step orchestration: each step calls one
// MCP tool, with subsequent steps able to reference prior step outputs
// in their argument templates. PollUntilField/Value turn a step into a
// polling loop (for async jobs); type:"download" turns a step into an
// HTTP download sink that writes to a CLI-local --output flag. When
// Pipeline is non-empty, dispatch ignores the parent toolOverrides map
// key (no single "primary tool"); the executor walks the steps in
// order. CLI surface (CLIName / Group / Flags) still comes from the
// parent override; flags can be marked PipelineLocal=true to be
// consumed by the executor without being forwarded to MCP tools.
//
// See internal/compat/pipeline.go for the executor + envelope examples.
Pipeline []PipelineStep `json:"pipeline,omitempty"`
}
// PipelineStep declares one step in a multi-step CLIToolOverride.Pipeline.
// Templates supported in Args / DownloadURLField:
//
// $flag.<aliasName> — value of the user's CLI flag whose alias is
// <aliasName> (resolved at dispatch time).
// $step.<idx>.<dotPath> — field from a prior step's response, e.g.
// "$step.0.jobId" or "$step.1.result.url".
// literal value — passed through unchanged.
type PipelineStep struct {
// Type controls dispatch. Empty / "call" invokes Tool as an MCP tool.
// "download" treats this step as an HTTP GET sink (no MCP tool is
// invoked); URL is resolved from DownloadURLField.
Type string `json:"type,omitempty"`
// Tool is the MCP tool name to invoke for type=="call".
Tool string `json:"tool,omitempty"`
// Args maps MCP tool parameter names to template strings.
Args map[string]string `json:"args,omitempty"`
// PollUntilField, when non-empty (with PollUntilValue), turns this
// step into a polling loop: invoke repeatedly with the same Args
// until response[<PollUntilField>] equals PollUntilValue (string
// compare). Use for async-job patterns where a status field
// transitions to a terminal value (e.g. "done" / "succeeded").
PollUntilField string `json:"pollUntilField,omitempty"`
PollUntilValue string `json:"pollUntilValue,omitempty"`
PollIntervalSec int `json:"pollIntervalSec,omitempty"` // default 2 when polling
PollTimeoutSec int `json:"pollTimeoutSec,omitempty"` // default 300 when polling
// DownloadURLField (type=="download") is a $step.X.field template
// that resolves to an HTTP URL. The body is fetched via GET and
// written to the path given by OutputFlag's value. If OutputFlag's
// value is empty, the URL is printed to stdout for the user.
DownloadURLField string `json:"downloadURLField,omitempty"`
// OutputFlag (type=="download") names the CLI flag (alias) whose
// user-supplied value is the local destination path. When the path
// is a directory, the filename is inferred from the URL's basename.
OutputFlag string `json:"outputFlag,omitempty"`
}
// CLIFlagOverride describes how to map an MCP parameter to a CLI flag.
@@ -261,6 +319,12 @@ type CLIFlagOverride struct {
// Resolution comes from edition.Hooks.RuntimeDefaults; open-source core
// only recognises the placeholder set. See schema v3 §2.3.
RuntimeDefault string `json:"runtimeDefault,omitempty"`
// PipelineLocal, when true, marks this flag as CLI-side only — its
// value is consumed by the pipeline executor (e.g. as an HTTP
// download destination) and NOT forwarded to any MCP tool's params.
// Only meaningful when the enclosing CLIToolOverride.Pipeline is set.
// Use for flags like `--output` that describe local destination paths.
PipelineLocal bool `json:"pipelineLocal,omitempty"`
}
type CLITool struct {