Files
dingtalk-workspace-cli/internal/output/formatter.go
T
修雨 c4952d0207 feat(output): add -f ndjson and -f csv (#252) (#259)
* feat(output): add -f ndjson (implemented) and scaffold -f csv (#252)

`larksuite/cli` exposes --format ndjson / csv; dws only had json/table/raw/
pretty. This adds both as recognised global formats:

- ndjson: fully implemented (internal/output/ndjson.go) — top-level arrays
  and well-known list wrappers ({items|results|data|records|...}) emit one
  compact JSON document per line; anything else degrades to a single line.
  Streaming-friendly counterpart to `-f json`.
- csv: scaffolded (internal/output/csv.go) — writeCSV currently returns a
  clear "not implemented (#252)" error rather than silently degrading to
  JSON. A TODO block spells out the planned implementation (reuse the table
  renderer's extractRowsFromMap/rowsFromSlice flattening + encoding/csv).

Wired through Write() and normalizeFormat(); --format help string now lists
ndjson (csv to be added when writeCSV lands). Tests cover ndjson rendering
(array / wrapped-list / scalar) and pin the csv not-implemented contract.

Skeleton PR for #252 — ndjson is shippable as-is; csv is left for a
follow-up commit on this branch.

* feat(output): implement -f csv (#252)

Completes the CSV half of the format work. writeCSV mirrors the shape
decisions `-f table` already makes (reuses normalizePayload /
unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice / formatValue) so
columns and value flattening are consistent between the two formats:

- a list of objects (bare or wrapped under items/results/data/records/...) →
  header row + one row per element; union of keys sorted; missing values are
  empty cells; nested objects/arrays render as compact JSON in the cell;
  sibling metadata of the list (total, hasMore, ...) is dropped.
- a single object → two-column key,value CSV (keys sorted).
- a non-uniform list / scalar → single-column `value` CSV.
- empty / nil → empty document.

encoding/csv.Writer handles RFC-4180 quoting (commas, quotes, newlines);
cells go through formatValue (also strips terminal control sequences, same
as the table renderer). `--fields` projection composes for free since
WriteFiltered applies SelectFields before Write.

--format help now lists csv; FormatCSV doc comment dropped the WIP marker;
the not-implemented test is replaced with real coverage (list with
comma/CJK/nested-array-as-JSON, wrapped-list-with-metadata, single object,
scalar) plus a --fields composition test.

* feat(output): broadcast list metadata as trailing columns in -f csv (#252)

Per review preference: instead of dropping the list's sibling metadata
(total, hasMore, ...) when rendering {records:[...], total:N} as CSV, append
each meta key as a trailing column repeated on every row, so a CSV consumer
never silently loses it (CSV has no "footer table" the way the table
renderer does). Meta keys colliding with a data column are skipped; an empty
list still emits the header (data + meta) plus one row of empty data cells
carrying the meta values. New broadcastMeta helper; doc comment + tests
updated (incl. an empty-list-with-metadata case).

* feat(output): recognise real DingTalk envelope keys in -f csv/ndjson/table

之前 -f csv 和 -f ndjson 的 list 检测白名单只认 items/results/data/list/
records/tools/servers/products,但真实钉钉响应用的是 result(单数直接数组
或一层包裹)/documents/emailAccounts/todoCards/events/messages,导致大
部分 list 命令的 csv 输出退化成 key,value 二列、ndjson 退化成整包一行。

修复点:

1. preferredListKeys 扩展加上真实 envelope key(result/documents/
   emailAccounts/todoCards/events/messages),并把它升级为 csv/table/
   ndjson 共享的"单一事实源"——filter.go 的 findDataList 不再维护自己
   的本地副本,直接复用这个列表。
2. extractRowsFromMap 改为委托 findDataList,自动获得"一层深度"的
   wrapper 支持({result: {todoCards: [...]}} 这种 envelope 现在能识
   别)。meta 合并:outer + inner 双层 sibling 拉通,outer 同名 key
   优先(避免 inner 把外层 success/total 等覆盖掉)。
3. writeTableish 和 writeCSV 调整 unwrapPrimaryObject 和
   extractRowsFromMap 的优先级——先试 list 检测,没命中再走 unwrap,
   避免 {result: {todoCards: [...]}} 被 unwrap 剥掉外层后直接走
   key,value 分支。
4. findDataList 允许"空数组+preferred key" 命中,保留原来"空 list +
   metadata 仍渲染为表格 + meta 广播一行"的行为。

新增 TestTabularDetectsRealDingTalkEnvelopes:四个真实 envelope 形态
(contact user search 的 result 直接数组、doc search 的 documents 顶层、
mail mailbox list 的 emailAccounts、todo task list 的 result.todoCards
一层深度)验证 ndjson 行数和 csv 表头都符合预期。

真接口回归(已登录态跑过):
- dws contact user search -f csv  → name/userId 列正常出表
- dws todo task list -f ndjson    → 20 行一条任务,可 jq -r .subject 直接管
- dws doc search -f csv           → 10 行 + nextPageToken 等 meta 广播尾列
- dws schema -f csv               → 无回归

Closes part of #252 follow-up.
2026-05-11 22:15:59 +08:00

577 lines
14 KiB
Go

// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package output
import (
"encoding/json"
"fmt"
"io"
"sort"
"strings"
apperrors "github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/errors"
"github.com/DingTalk-Real-AI/dingtalk-workspace-cli/internal/executor"
"github.com/spf13/cobra"
"github.com/spf13/pflag"
)
type Format string
const (
FormatJSON Format = "json"
FormatTable Format = "table"
FormatRaw Format = "raw"
FormatPretty Format = "pretty"
// FormatNDJSON emits one JSON object per line — friendly for streaming /
// piping list results into downstream tools. See ndjson.go.
FormatNDJSON Format = "ndjson"
// FormatCSV emits RFC-4180 comma-separated values for list-shaped results —
// friendly for spreadsheets and non-technical consumers. See csv.go.
FormatCSV Format = "csv"
)
// preferredListKeys is the shared allow-list of keys whose array values are
// treated as the "data list" by all tabular formatters (-f table / csv /
// ndjson). It is the single source of truth — findDataList in filter.go
// reuses it. When adding a new key, prefer real envelope keys observed in
// production responses over speculative future names.
var preferredListKeys = []string{
// Generic well-known list keys.
"value", "items", "results", "data", "list", "records",
"tools", "servers", "products",
// Envelope keys observed in real DingTalk responses.
"result", "documents", "emailAccounts", "todoCards", "events", "messages",
}
func ResolveFormat(cmd *cobra.Command, fallback Format) Format {
if cmd == nil {
return fallback
}
for _, flags := range []*pflag.FlagSet{cmd.Flags(), cmd.InheritedFlags()} {
if format, ok := formatFromFlagSet(flags, fallback); ok {
return format
}
}
if root := cmd.Root(); root != nil {
if format, ok := formatFromFlagSet(root.PersistentFlags(), fallback); ok {
return format
}
}
return fallback
}
func WriteCommandPayload(cmd *cobra.Command, payload any, fallback Format) error {
if cmd == nil {
return Write(io.Discard, fallback, payload)
}
return WriteFiltered(
cmd.OutOrStdout(),
ResolveFormat(cmd, fallback),
payload,
ResolveFields(cmd),
ResolveJQ(cmd),
)
}
func Write(w io.Writer, format Format, payload any) error {
payload = unwrapCompatRuntimePayload(payload)
switch format {
case FormatJSON:
return WriteJSON(w, payload)
case FormatRaw:
return writeRaw(w, payload)
case FormatTable:
return writeTableish(w, payload)
case FormatPretty:
return writePretty(w, payload)
case FormatNDJSON:
return writeNDJSON(w, payload)
case FormatCSV:
return writeCSV(w, payload)
default:
return WriteJSON(w, payload)
}
}
func unwrapCompatRuntimePayload(payload any) any {
result, ok := payload.(executor.Result)
if !ok {
return payload
}
if !result.Invocation.Implemented {
return payload
}
switch result.Invocation.Kind {
case "compat_invocation", "helper_invocation":
content, ok := result.Response["content"]
if ok {
return content
}
}
return payload
}
func formatFromFlagSet(flags *pflag.FlagSet, fallback Format) (Format, bool) {
if flags == nil {
return fallback, false
}
flag := flags.Lookup("format")
if flag == nil {
return fallback, false
}
value, err := flags.GetString("format")
if err != nil {
return fallback, false
}
return normalizeFormat(value, fallback), true
}
func normalizeFormat(raw string, fallback Format) Format {
switch strings.ToLower(strings.TrimSpace(raw)) {
case "", string(fallback):
return fallback
case string(FormatJSON):
return FormatJSON
case string(FormatRaw):
return FormatRaw
case string(FormatTable):
return FormatTable
case string(FormatPretty):
return FormatPretty
case string(FormatNDJSON):
return FormatNDJSON
case string(FormatCSV):
return FormatCSV
default:
return fallback
}
}
// WriteFiltered applies field selection and/or jq filtering before
// writing the payload. If jq is non-empty, the jq result is written
// directly (bypassing format). If fields is non-empty, the payload
// is filtered to those fields before normal output.
func WriteFiltered(w io.Writer, format Format, payload any, fields, jq string) error {
payload = unwrapCompatRuntimePayload(payload)
if strings.TrimSpace(jq) != "" {
return ApplyJQ(w, payload, strings.TrimSpace(jq))
}
if strings.TrimSpace(fields) != "" {
fieldList := strings.Split(fields, ",")
payload = SelectFields(payload, fieldList)
}
return Write(w, format, payload)
}
// ResolveFields extracts the --fields flag value from the command.
// It ensures that we do not mistakenly grab a business parameter also named "fields"
// by matching the flag's usage string against the global root definition.
func ResolveFields(cmd *cobra.Command) string {
if cmd == nil {
return ""
}
rootFlags := rootPersistentFlags(cmd)
if rootFlags == nil {
return ""
}
globalFlag := rootFlags.Lookup("fields")
if globalFlag == nil {
return ""
}
for _, flags := range []*pflag.FlagSet{
cmd.Flags(),
cmd.InheritedFlags(),
rootFlags,
} {
if flags == nil {
continue
}
if f := flags.Lookup("fields"); f != nil && f.Changed {
// To avoid collision with business flags (e.g. table create --fields),
// verify this flag shares the same usage string as the global one.
if f.Usage == globalFlag.Usage {
if v, err := flags.GetString("fields"); err == nil {
return v
}
}
}
}
return ""
}
// ResolveJQ extracts the --jq flag value from the command. It ensures
// that we only grab the global output filter, not a similarly named business parameter.
func ResolveJQ(cmd *cobra.Command) string {
if cmd == nil {
return ""
}
rootFlags := rootPersistentFlags(cmd)
if rootFlags == nil {
return ""
}
globalFlag := rootFlags.Lookup("jq")
if globalFlag == nil {
return ""
}
for _, flags := range []*pflag.FlagSet{
cmd.Flags(),
cmd.InheritedFlags(),
rootFlags,
} {
if flags == nil {
continue
}
if f := flags.Lookup("jq"); f != nil && f.Changed {
if f.Usage == globalFlag.Usage {
if v, err := flags.GetString("jq"); err == nil {
return v
}
}
}
}
return ""
}
func rootPersistentFlags(cmd *cobra.Command) *pflag.FlagSet {
if root := cmd.Root(); root != nil {
return root.PersistentFlags()
}
return nil
}
// WriteJSON marshals payload as indented JSON and writes it to w.
func WriteJSON(w io.Writer, payload any) error {
data, err := json.MarshalIndent(payload, "", " ")
if err != nil {
return apperrors.NewInternal("failed to encode command output as JSON")
}
_, err = fmt.Fprintln(w, string(data))
return err
}
func writeRaw(w io.Writer, payload any) error {
if text, ok := payload.(string); ok {
_, err := fmt.Fprintln(w, SanitizeForTerminal(text))
return err
}
data, err := json.Marshal(payload)
if err != nil {
return apperrors.NewInternal("failed to encode raw command output")
}
_, err = fmt.Fprintln(w, SanitizeForTerminal(string(data)))
return err
}
func writeTableish(w io.Writer, payload any) error {
normalized, err := normalizePayload(payload)
if err != nil {
return err
}
switch typed := normalized.(type) {
case map[string]any:
// Try table extraction first so wrappers around list payloads
// (e.g. {result: {todoCards: [...]}}) render as a table instead
// of being peeled by unwrapPrimaryObject and degraded to key/
// value rows. unwrapPrimaryObject remains the fallback for
// single-object wrappers like {invocation: {kind, params, ...}}.
if headers, rows, meta, ok := extractRowsFromMap(typed); ok {
if err := writeTable(w, headers, rows); err != nil {
return err
}
if len(meta) > 0 {
if _, err := fmt.Fprintln(w); err != nil {
return err
}
return writeKeyValues(w, meta)
}
return nil
}
if inner, ok := unwrapPrimaryObject(typed); ok {
return writeKeyValues(w, inner)
}
return writeKeyValues(w, typed)
case []any:
if headers, rows, ok := rowsFromSlice(typed); ok {
return writeTable(w, headers, rows)
}
return writeRaw(w, normalized)
default:
return writeRaw(w, normalized)
}
}
func normalizePayload(payload any) (any, error) {
if payload == nil {
return nil, nil
}
if text, ok := payload.(string); ok {
return text, nil
}
data, err := json.Marshal(payload)
if err != nil {
return nil, apperrors.NewInternal("failed to normalize command output")
}
var normalized any
if err := json.Unmarshal(data, &normalized); err != nil {
return nil, apperrors.NewInternal("failed to decode normalized command output")
}
return normalized, nil
}
func unwrapPrimaryObject(payload map[string]any) (map[string]any, bool) {
if len(payload) != 1 {
return nil, false
}
for _, key := range []string{"invocation", "response", "result", "data"} {
value, ok := payload[key]
if !ok {
continue
}
object, ok := value.(map[string]any)
if ok {
return object, true
}
}
return nil, false
}
// extractRowsFromMap finds the data list inside a wrapper map and returns it
// as (headers, rows, meta). It delegates the search to findDataList so the
// detection rules stay aligned with -f ndjson: top-level under a preferred
// key, or one level deep under {result|response|data}. Meta is built from
// every sibling of the list — at both the outer and inner level when the
// list sits one level deep — so callers like the table renderer's footer and
// the csv broadcastMeta path see the same key set.
func extractRowsFromMap(payload map[string]any) ([]string, [][]string, map[string]any, bool) {
loc := findDataList(payload)
if loc == nil {
return nil, nil, nil, false
}
headers, rows, ok := rowsFromSlice(loc.list)
if !ok {
return nil, nil, nil, false
}
meta := make(map[string]any)
if loc.outerKey == "" {
for k, v := range payload {
if k == loc.innerKey {
continue
}
meta[k] = v
}
} else {
for k, v := range payload {
if k == loc.outerKey {
continue
}
meta[k] = v
}
if inner, ok := payload[loc.outerKey].(map[string]any); ok {
for k, v := range inner {
if k == loc.innerKey {
continue
}
if _, exists := meta[k]; exists {
// Outer wins on key collision so users see the wrapper-level
// sibling rather than a clobbered inner one.
continue
}
meta[k] = v
}
}
}
return headers, rows, meta, true
}
func rowsFromSlice(items []any) ([]string, [][]string, bool) {
if len(items) == 0 {
return []string{"value"}, [][]string{}, true
}
allMaps := true
keys := make(map[string]struct{})
for _, item := range items {
rowMap, ok := item.(map[string]any)
if !ok {
allMaps = false
break
}
for key := range rowMap {
keys[key] = struct{}{}
}
}
if allMaps {
headers := sortedKeys(keys)
rows := make([][]string, 0, len(items))
for _, item := range items {
rowMap := item.(map[string]any)
row := make([]string, 0, len(headers))
for _, key := range headers {
row = append(row, formatValue(rowMap[key]))
}
rows = append(rows, row)
}
return headers, rows, true
}
rows := make([][]string, 0, len(items))
for _, item := range items {
rows = append(rows, []string{formatValue(item)})
}
return []string{"value"}, rows, true
}
func writeKeyValues(w io.Writer, payload map[string]any) error {
keys := make([]string, 0, len(payload))
maxWidth := 0
for key := range payload {
keys = append(keys, key)
if width := runeWidth(key); width > maxWidth {
maxWidth = width
}
}
sort.Strings(keys)
if maxWidth > 24 {
maxWidth = 24
}
for _, key := range keys {
if _, err := fmt.Fprintf(w, "%-*s %s\n", maxWidth, key, formatValue(payload[key])); err != nil {
return err
}
}
return nil
}
func writeTable(w io.Writer, headers []string, rows [][]string) error {
widths := make([]int, len(headers))
for i, header := range headers {
widths[i] = runeWidth(header)
}
for _, row := range rows {
for i, cell := range row {
if i >= len(widths) {
continue
}
if width := runeWidth(cell); width > widths[i] {
widths[i] = width
}
}
}
for i := range widths {
if widths[i] > 60 {
widths[i] = 60
}
}
for i, header := range headers {
if i > 0 {
if _, err := io.WriteString(w, " "); err != nil {
return err
}
}
if _, err := fmt.Fprintf(w, "%-*s", widths[i], truncate(header, widths[i])); err != nil {
return err
}
}
if _, err := fmt.Fprintln(w); err != nil {
return err
}
for i, width := range widths {
if i > 0 {
if _, err := io.WriteString(w, " "); err != nil {
return err
}
}
if _, err := io.WriteString(w, strings.Repeat("-", width)); err != nil {
return err
}
}
if _, err := fmt.Fprintln(w); err != nil {
return err
}
for _, row := range rows {
for i, cell := range row {
if i >= len(widths) {
continue
}
if i > 0 {
if _, err := io.WriteString(w, " "); err != nil {
return err
}
}
if _, err := fmt.Fprintf(w, "%-*s", widths[i], truncate(cell, widths[i])); err != nil {
return err
}
}
if _, err := fmt.Fprintln(w); err != nil {
return err
}
}
return nil
}
func sortedKeys(keys map[string]struct{}) []string {
out := make([]string, 0, len(keys))
for key := range keys {
out = append(out, key)
}
sort.Strings(out)
return out
}
func formatValue(value any) string {
switch typed := value.(type) {
case nil:
return ""
case string:
return SanitizeForTerminal(typed)
default:
data, err := json.Marshal(typed)
if err != nil {
return SanitizeForTerminal(fmt.Sprintf("%v", typed))
}
return SanitizeForTerminal(string(data))
}
}
func truncate(s string, maxWidth int) string {
if runeWidth(s) <= maxWidth {
return s
}
runes := []rune(s)
if len(runes) > maxWidth-1 {
return string(runes[:maxWidth-1]) + "…"
}
return s
}
func runeWidth(s string) int {
width := 0
for _, r := range s {
if r >= 0x2E80 && r <= 0x9FFF {
width += 2
continue
}
width++
}
return width
}