Compare commits

...
Author SHA1 Message Date
修雨 d5dbefb27e docs: sync outdated references and add audit-log guide
- architecture.md: rewrite to reflect current plugin-based structure
  (removed market/discovery/ir/generator/compat, added helpers/plugin/audit/pat)
- automation.md: update task routing and repository map to current packages
- command-index.md: fix source citation (internal/plugin replaces removed compat pipeline)
- connect-daemon-service.md: update command path from 'dws devapp robot connect'
  to 'dws dev connect' (7 occurrences + launchd plist args)
- audit-log.md: new doc covering event format, env vars, CLI commands,
  hash chain verification, remote forwarding, and redaction levels
2026-07-06 15:38:57 +08:00
修雨 1c1abf3069 style(helpers): fix gofmt formatting for devapp_connect and connect_daemon
Import ordering and struct field alignment were off since #548,
causing the CI format check to fail on main.
2026-07-06 15:25:24 +08:00
7 changed files with 308 additions and 67 deletions
+31 -16
View File
@@ -4,23 +4,38 @@
## High-Level Flow
1. `internal/market` fetches the registry and server metadata.
2. `internal/discovery` resolves runtime server capabilities and caches results.
3. `internal/ir` normalizes discovery output into one canonical tool catalog.
4. `internal/cli` and `internal/app` mount that catalog into the public Cobra command tree.
5. `internal/transport` executes MCP JSON-RPC calls and `internal/output` formats responses.
1. `cmd` is the CLI entrypoint, invoking `internal/app` to build the root Cobra command tree.
2. `internal/app` wires static utility commands (`auth`, `audit`, `schema`, `completion`) and dynamically loads product commands via `internal/plugin`.
3. `internal/helpers` contains the main command handlers for all product surfaces (`dev`, `chat`, `calendar`, `contact`, `aitable`, etc.).
4. `internal/executor` and `internal/transport` execute MCP JSON-RPC calls; `internal/output` formats responses.
5. `internal/auth` manages login state, PAT tokens, and agent-code detection.
## Repository Structure
- `cmd`: CLI entrypoint
- `internal/app`: root command wiring and static utility commands
- `internal/discovery`, `internal/market`, `internal/transport`: runtime discovery and execution
- `internal/ir`: canonical intermediate representation for discovered tools
- `internal/generator`: docs, schema, and skill generation pipeline
- `internal/compat`, `internal/helpers`: legacy-compatible overlays and helper commands
- `skills/`: bundled agent skills source and generated skill docs
- `test/`: CLI, compatibility, integration, contract, and script tests
## Public Repository Contract
This repository ships source, docs, tests, packaging templates, and install scripts. Generated or release-only artifacts are produced by repository scripts and are not required to exist in a clean checkout unless explicitly committed as part of a release workflow.
- `internal/app`: root command wiring, static utility commands, and plugin loading
- `internal/helpers`: product command handlers (dev, chat, calendar, contact, etc.)
- `internal/plugin`: plugin-based dynamic command loader
- `internal/cli`: catalog types and endpoint loader (static endpoint mode)
- `internal/executor`: invocation dispatch and result handling
- `internal/transport`: MCP HTTP client and request signing
- `internal/auth`: login, token management, agent-code detection, identity
- `internal/audit`: user operation audit log (JSONL, hash chain, forwarding)
- `internal/errors`: structured error model with categories and hints
- `internal/keychain`: OS keychain integration for credential storage
- `internal/security`: endpoint allowlist and domain trust
- `internal/safety`: runtime safety checks (confirm prompts, dry-run guards)
- `internal/cobracmd`: shared Cobra command builders
- `internal/pat`: PAT (Personal Access Token) authorization flow
- `internal/output`: response formatting (json, table, raw, pretty)
- `internal/logging`: structured logging and argument sanitization
- `internal/tui`: terminal UI helpers
- `internal/recovery`: panic recovery and graceful degradation
- `pkg/configmeta`: environment variable registry and documentation
- `pkg/config`: configuration constants and paths
- `pkg/edition`: edition detection (oss vs enterprise)
- `pkg/mcptypes`: MCP protocol type definitions
- `skills/`: bundled agent skills (mono/ and multi/ layouts)
- `test/`: CLI, integration, contract, unit, and skill E2E tests
- `scripts/`: install scripts, policy checks, and CI helpers
- `envelope/`: pre-built discovery payloads for offline use
+227
View File
@@ -0,0 +1,227 @@
# 审计日志(Audit Log)
DWS 自动记录每次 MCP HTTP 调用的审计事件,用于合规追溯和安全审查。默认启用,无需额外配置。
## 功能特性
- **自动记录**:每次命令执行产生一条 JSONL 审计事件
- **按天轮转**:日志文件按日期分割(`audit-YYYYMMDD.jsonl`),默认留存 90 天
- **防篡改**:L1 哈希链(sha256),每条事件链接前一条的 hash,可验证完整性
- **远端转发**:支持 POST 到外部 SIEM 或审计平台
- **三级脱敏**:转发时可按 none/hashed/minimal 脱敏敏感字段
- **CLI 命令**:内置 `dws audit tail/export/verify` 查看和验证
## 环境变量
| 变量 | 默认值 | 说明 |
|------|--------|------|
| `DWS_AUDIT` | 启用 | 设 `0`/`false`/`off` 关闭审计 |
| `DWS_AUDIT_DIR` | `~/.dws/audit` | 审计日志目录 |
| `DWS_AUDIT_RETENTION_DAYS` | `90` | 日志留存天数 |
| `DWS_AUDIT_FORWARD_URL` | (空) | 远端转发 URL(POST JSON) |
| `DWS_AUDIT_FORWARD_TOKEN` | (空) | 远端转发 Bearer Token |
| `DWS_AUDIT_FORWARD_REDACT` | `none` | 转发脱敏级别:`none`/`hashed`/`minimal` |
## 审计事件格式
每条事件是一行 JSON,字段如下:
```json
{
"ts": "2026-07-06T10:59:06+08:00",
"execution_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"agent_id": "agent-xxx",
"actor": {
"user_id": "525018",
"name": "胡奕舟",
"corp_id": "ding8196cd9a2b2405da24f2f5cc6abecb85",
"corp_name": "钉钉"
},
"product": "calendar",
"command": "list_events",
"endpoint": "https://api.dingtalk.com/v1.0/calendar/users/xxx/calendars/primary/events",
"params_summary": "maxResults=20",
"result": "success",
"error_category": "",
"error_reason": "",
"duration_ms": 234,
"cli_version": "1.0.47",
"os": "darwin",
"arch": "arm64",
"prev_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"hash": "9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08"
}
```
**字段说明**:
- `ts`:事件时间戳(RFC3339)
- `execution_id`:本次命令执行的唯一 ID
- `agent_id`:Agent 标识(若有)
- `actor`:执行者信息(从登录态获取)
- `product`:调用的产品(如 `calendar`、`chat`、`contact`)
- `command`:调用的命令(如 `list_events`、`send_message`)
- `endpoint`:实际请求的 API 端点(已脱敏)
- `params_summary`:参数摘要(已脱敏)
- `result`:`success` 或 `error`
- `error_category` / `error_reason`:错误分类和原因(成功时为空)
- `duration_ms`:命令执行耗时(毫秒)
- `prev_hash` / `hash`:哈希链字段,用于防篡改验证
## CLI 命令
### 查看最近日志
```bash
# 查看最近 20 条审计事件(默认)
dws audit tail
# 查看最近 50 条
dws audit tail -n 50
```
输出示例:
```
2026-07-06 10:59:06 calendar list_events user=525018 result=success 234ms
2026-07-06 10:58:42 chat send_message user=525018 result=success 156ms
2026-07-06 10:57:15 contact search user=525018 result=success 89ms
```
### 导出日志
```bash
# 导出最近 7 天的 JSONL
dws audit export --since 2026-06-29 --until 2026-07-06 --format jsonl > audit-7d.jsonl
# 导出为 CSV(方便 Excel 打开)
dws audit export --since 2026-07-01 --format csv > audit-july.csv
```
### 验证哈希链
```bash
# 验证当前最新日志文件的哈希链完整性
dws audit verify
# 验证指定文件
dws audit verify --file ~/.dws/audit/audit-20260706.jsonl
```
输出:
```
✓ 哈希链完整:234 条事件全部校验通过
```
或:
```
✗ 哈希链断裂:第 156 条事件的 hash 不匹配
```
## 哈希链防篡改
每条事件的 `hash` 字段由以下公式计算:
```
hash = sha256(prev_hash + event_json_without_hash_fields)
```
- 首条事件的 `prev_hash` 为空字符串
- 后续事件的 `prev_hash` = 前一条的 `hash`
- 任何对历史事件的篡改都会导致后续所有 hash 失效
**验证流程**:
1. 读取日志文件,逐行解析
2. 对每条事件,移除 `prev_hash` 和 `hash` 字段,重新序列化
3. 用前一条的 hash + 当前事件 JSON 计算新 hash
4. 对比计算结果与文件中记录的 hash
5. 全部匹配 = 完整;某条不匹配 = 被篡改
## 远端转发
设置 `DWS_AUDIT_FORWARD_URL` 后,每条审计事件会异步 POST 到指定端点:
```bash
export DWS_AUDIT_FORWARD_URL=https://siem.example.com/ingest/audit
export DWS_AUDIT_FORWARD_TOKEN=your-bearer-token # 可选
```
**请求格式**:
```http
POST /ingest/audit HTTP/1.1
Host: siem.example.com
Authorization: Bearer your-bearer-token
Content-Type: application/json
{"ts":"2026-07-06T10:59:06+08:00","product":"calendar",...}
```
**超时与重试**:3 秒超时,失败不阻塞命令执行(best-effort),不自动重试。
## 脱敏分级
转发时可通过 `DWS_AUDIT_FORWARD_REDACT` 控制脱敏级别:
| 级别 | 行为 | 适用场景 |
|------|------|----------|
| `none`(默认) | 原样转发,不脱敏 | 内部审计平台 |
| `hashed` | actor.name 哈希化,params_summary 脱敏 | 跨部门共享 |
| `minimal` | 仅保留 ts/product/command/result/duration_ms,移除 actor/endpoint/params | 对外合规报告 |
**示例**:
```bash
# 最小化脱敏(仅保留元数据)
export DWS_AUDIT_FORWARD_REDACT=minimal
export DWS_AUDIT_FORWARD_URL=https://compliance.example.com/audit
```
## 常见问题
### Q: 审计日志占多少磁盘?
典型场景(每天 100 条命令)约 50KB/天,90 天约 4.5MB。日志文件是 JSONL 纯文本,gzip 压缩后约 1/5。
### Q: 关闭审计会影响性能吗?
设置 `DWS_AUDIT=0` 后,审计模块不初始化,零开销。默认启用时,每条事件写入耗时 <1ms(异步磁盘 IO)。
### Q: 哈希链断了怎么办?
可能原因:
1. 手动编辑过日志文件
2. 磁盘损坏
3. 并发写入导致顺序错乱(罕见)
**处理**:
- 备份当前日志
- 用 `dws audit verify` 定位断裂位置
- 从断裂点之后的事件可继续验证(前缀已不可信)
### Q: 如何清理旧日志?
自动清理:`DWS_AUDIT_RETENTION_DAYS=90`(默认),超过 90 天的文件在下次启动时 best-effort 删除。
手动清理:
```bash
# 删除 2026 年 6 月之前的日志
rm ~/.dws/audit/audit-202605*.jsonl
```
## 技术实现
- **核心包**:`internal/audit`
- **集成点**:`internal/app/runner.go` 的 `executeInvocation` 方法(defer 调用 `emitAudit`)
- **身份获取**:从 `auth.LoadTokenData` 读取当前登录用户
- **参数脱敏**:调用 `logging.SanitizeArguments`(与现有日志脱敏逻辑一致)
## 相关文档
- [环境变量参考](./reference.md)
- [架构概览](./architecture.md)
- [自动化与脚本](./automation.md)
+26 -25
View File
@@ -13,40 +13,44 @@ repository root while preserving repo-local guidance for automation.
## Project Snapshot
- `dws` is a Go-based DingTalk Workspace CLI and MCP runtime bridge.
- One internal Tool IR drives canonical CLI, schema, docs, skills, and snapshots.
- Compatibility and helper surfaces are overlays, not the canonical truth.
- Product commands are loaded dynamically via `internal/plugin` from bundled descriptors.
- Command handlers live in `internal/helpers`; runtime execution flows through `internal/executor` and `internal/transport`.
## Repository Map
- `cmd`: public CLI entrypoint
- `internal/app`: root command wiring and command tree mount points
- `internal/discovery`, `internal/market`, `internal/transport`: runtime discovery and MCP transport
- `internal/generator`: CLI/schema/docs/skills generation pipeline
- `internal/compat`, `internal/helpers`: legacy-compatible aliases and helper commands
- `internal/app`: root command wiring, static utility commands, plugin loading
- `internal/helpers`: product command handlers (dev, chat, calendar, contact, etc.)
- `internal/plugin`: plugin-based dynamic command loader
- `internal/cli`: catalog types and static endpoint loader
- `internal/executor`: invocation dispatch and result handling
- `internal/transport`: MCP HTTP client and request signing
- `internal/auth`: login, token management, agent-code detection
- `internal/audit`: user operation audit log
- `internal/errors`: structured error model with categories and hints
- `internal/keychain`: OS keychain integration for credential storage
- `internal/security`: endpoint allowlist and domain trust
- `internal/pat`: PAT (Personal Access Token) authorization flow
- `docs/`: public architecture and reference docs
- `hack/`: developer-only helper commands not shipped as public binaries
- `scripts/`: build, test, lint, packaging, and policy checks
- `test/`: integration, contract, compatibility, and script validation suites
- `test/`: CLI, integration, contract, unit, and skill E2E test suites
## Task Routing
- Add or fix a command path: start from `internal/app` and the related module under `internal/*`
- Discovery or protocol issues: inspect `internal/discovery`, `internal/market`, `internal/transport`
- Generated output drift: inspect `internal/generator` and run drift checks
- Legacy behavior mismatch: inspect `internal/compat` and `test/cli_compat`
- Failure or degraded mode: inspect `internal/discovery`, `internal/errors`
- Add or fix a command path: start from `internal/helpers` (handler implementations) or `internal/app` (command tree wiring)
- Protocol or transport issues: inspect `internal/transport`
- Auth or login issues: inspect `internal/auth`, `internal/pat`, `internal/keychain`
- Error message or category issues: inspect `internal/errors`
- Audit log issues: inspect `internal/audit`
- Plugin loading or command surface: inspect `internal/plugin`
- Failure or degraded mode: inspect `internal/errors`, `internal/recovery`
## Generated Artifacts
## Policy Checks
Prefer editing source logic instead of generated files directly.
When command surface or plugin descriptors change, run:
- Generated-heavy paths:
- `docs/generated/`
- `skills/generated/`
- `test/golden/generated_outputs/`
- When generator or command surface changes, run:
- `./scripts/policy/check-generated-drift.sh`
- `./scripts/policy/check-command-surface.sh --strict`
- `./scripts/policy/check-command-surface.sh --strict`
- `./scripts/policy/check-open-source-assets.sh`
## Common Commands
@@ -55,9 +59,6 @@ make build
make test
make lint
./scripts/dev/ci-local.sh
./scripts/policy/check-generated-drift.sh
./scripts/policy/check-command-surface.sh --strict
./scripts/policy/check-open-source-assets.sh
git diff --check
```
+2 -3
View File
@@ -2,12 +2,11 @@
Every runtime command the `dws` CLI exposes when loaded with the **pre** environment configuration.
- **Source**: `dws-wukong/envelope/channel/open/pre/config.json`
- **Products**: 13
- **Total commands**: 160
- **Generated from**: `internal/compat.BuildDynamicCommands` rendering of the pre config — the same code path the CLI uses at runtime.
- **Generated from**: `internal/plugin` command descriptors — the same code path the CLI uses at runtime.
> Auto-generated. Edit `pre/config.json`, not this file.
> Auto-generated. Update plugin descriptors in `internal/plugin/`, not this file.
## Global flags
+8 -9
View File
@@ -1,6 +1,6 @@
# Running the connector as a 7x24 service
`dws devapp robot connect` keeps a DingTalk robot wired to a local agent over a
`dws dev connect` keeps a DingTalk robot wired to a local agent over a
Stream long-connection. By default it runs in the foreground and dies when the
terminal closes. For an unattended "digital employee" you have two options.
@@ -15,14 +15,14 @@ terminal closes. For an unattended "digital employee" you have two options.
```bash
# Detach into a background supervisor that restarts the connector if it crashes.
dws devapp robot connect --daemon \
dws dev connect --daemon \
--channel claudecode \
--unified-app-id <unifiedAppId>
# Inspect / stop / restart it (locate the daemon by unifiedAppId).
dws devapp robot connect status --unified-app-id <unifiedAppId>
dws devapp robot connect stop --unified-app-id <unifiedAppId>
dws devapp robot connect restart --unified-app-id <unifiedAppId>
dws dev connect status --unified-app-id <unifiedAppId>
dws dev connect stop --unified-app-id <unifiedAppId>
dws dev connect restart --unified-app-id <unifiedAppId>
```
- The parent prints the daemon pid and the log path, then exits.
@@ -60,8 +60,7 @@ and `REPLACE_UNIFIED_APP_ID`, then `launchctl load -w <path>`.
<key>ProgramArguments</key>
<array>
<string>/usr/local/bin/dws</string>
<string>devapp</string>
<string>robot</string>
<string>dev</string>
<string>connect</string>
<string>--channel</string>
<string>claudecode</string>
@@ -110,7 +109,7 @@ Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/dws devapp robot connect \
ExecStart=/usr/local/bin/dws dev connect \
--channel claudecode \
--unified-app-id REPLACE_UNIFIED_APP_ID
Restart=always
@@ -136,7 +135,7 @@ security warning to stderr. This form:
- exposes `clientSecret` to every user on the box via `ps -ef`;
- gets baked into launchd `ProgramArguments` / systemd `ExecStart`, which
makes rotation harder;
- means `dws devapp robot connect restart` cannot re-fetch credentials — you
- means `dws dev connect restart` cannot re-fetch credentials — you
must re-run the full command yourself.
Prefer `--unified-app-id`. Only fall back to the pair when you understand the
+13 -13
View File
@@ -26,12 +26,12 @@ import (
"syscall"
"time"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cobracmd"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/executor"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/logging"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/tui"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/pkg/config"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/cobracmd"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/executor"
"github.com/spf13/cobra"
)
@@ -61,13 +61,13 @@ const (
// supervisor pid plus enough context for `status` to report without re-deriving
// it (start time for uptime, log path, the dir key it was filed under).
type daemonState struct {
Pid int `json:"pid"`
StartUnix int64 `json:"startUnix"`
LogPath string `json:"logPath"`
DirKey string `json:"dirKey"`
ClientID string `json:"clientId,omitempty"`
UnifiedAppID string `json:"unifiedAppId,omitempty"`
Channel string `json:"channel,omitempty"`
Pid int `json:"pid"`
StartUnix int64 `json:"startUnix"`
LogPath string `json:"logPath"`
DirKey string `json:"dirKey"`
ClientID string `json:"clientId,omitempty"`
UnifiedAppID string `json:"unifiedAppId,omitempty"`
Channel string `json:"channel,omitempty"`
NotifyStaffID string `json:"notifyStaffId,omitempty"`
// Profile records the --profile selector the connector was started with,
// so `restart` re-fetches credentials against the same org instead of the
@@ -112,7 +112,7 @@ func connectDaemonDir(dirKey string) (string, error) {
func daemonPidPath(dir string) string { return filepath.Join(dir, "daemon.pid") }
func daemonStatePath(dir string) string { return filepath.Join(dir, "daemon-state.json") }
func daemonLogPath(dir string) string { return filepath.Join(dir, "daemon.log") }
func daemonLogPath(dir string) string { return filepath.Join(dir, "daemon.log") }
// writeDaemonState atomically persists the daemon state to daemon-state.json
// (persistent, survives supervisor exit) so restart/list can recover config.
@@ -641,9 +641,9 @@ func newDevAppRobotConnectStopCommand() *cobra.Command {
// platform — no secrets stored on disk.
func newDevAppRobotConnectRestartCommand() *cobra.Command {
cmd := &cobra.Command{
Use: "restart",
Short: "重启连接器守护进程(通过持久化的 unifiedAppId 重新拉取密钥,无需本地存密钥)",
Args: cobra.NoArgs,
Use: "restart",
Short: "重启连接器守护进程(通过持久化的 unifiedAppId 重新拉取密钥,无需本地存密钥)",
Args: cobra.NoArgs,
DisableAutoGenTag: true,
RunE: func(cmd *cobra.Command, _ []string) error {
dirKey, err := connectDaemonDirKeyFromFlags(cmd)
+1 -1
View File
@@ -677,7 +677,7 @@ func connectAgentOptionsFromCommand(cmd *cobra.Command) connectAgentOptions {
auditSheetTab = "Sheet1"
}
return connectAgentOptions{Model: model, WorkDir: workDir, Memory: memory,
Timeout: time.Duration(agentTimeoutSec) * time.Second,
Timeout: time.Duration(agentTimeoutSec) * time.Second,
ReplyCard: replyCard, CardTemplate: cardTemplate,
KnowledgeDir: knowledgeDir,
KnowledgeSource: knowledgeSource,