Files
修雨 c4952d0207 feat(output): add -f ndjson and -f csv (#252) (#259)
* feat(output): add -f ndjson (implemented) and scaffold -f csv (#252)

`larksuite/cli` exposes --format ndjson / csv; dws only had json/table/raw/
pretty. This adds both as recognised global formats:

- ndjson: fully implemented (internal/output/ndjson.go) — top-level arrays
  and well-known list wrappers ({items|results|data|records|...}) emit one
  compact JSON document per line; anything else degrades to a single line.
  Streaming-friendly counterpart to `-f json`.
- csv: scaffolded (internal/output/csv.go) — writeCSV currently returns a
  clear "not implemented (#252)" error rather than silently degrading to
  JSON. A TODO block spells out the planned implementation (reuse the table
  renderer's extractRowsFromMap/rowsFromSlice flattening + encoding/csv).

Wired through Write() and normalizeFormat(); --format help string now lists
ndjson (csv to be added when writeCSV lands). Tests cover ndjson rendering
(array / wrapped-list / scalar) and pin the csv not-implemented contract.

Skeleton PR for #252 — ndjson is shippable as-is; csv is left for a
follow-up commit on this branch.

* feat(output): implement -f csv (#252)

Completes the CSV half of the format work. writeCSV mirrors the shape
decisions `-f table` already makes (reuses normalizePayload /
unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice / formatValue) so
columns and value flattening are consistent between the two formats:

- a list of objects (bare or wrapped under items/results/data/records/...) →
  header row + one row per element; union of keys sorted; missing values are
  empty cells; nested objects/arrays render as compact JSON in the cell;
  sibling metadata of the list (total, hasMore, ...) is dropped.
- a single object → two-column key,value CSV (keys sorted).
- a non-uniform list / scalar → single-column `value` CSV.
- empty / nil → empty document.

encoding/csv.Writer handles RFC-4180 quoting (commas, quotes, newlines);
cells go through formatValue (also strips terminal control sequences, same
as the table renderer). `--fields` projection composes for free since
WriteFiltered applies SelectFields before Write.

--format help now lists csv; FormatCSV doc comment dropped the WIP marker;
the not-implemented test is replaced with real coverage (list with
comma/CJK/nested-array-as-JSON, wrapped-list-with-metadata, single object,
scalar) plus a --fields composition test.

* feat(output): broadcast list metadata as trailing columns in -f csv (#252)

Per review preference: instead of dropping the list's sibling metadata
(total, hasMore, ...) when rendering {records:[...], total:N} as CSV, append
each meta key as a trailing column repeated on every row, so a CSV consumer
never silently loses it (CSV has no "footer table" the way the table
renderer does). Meta keys colliding with a data column are skipped; an empty
list still emits the header (data + meta) plus one row of empty data cells
carrying the meta values. New broadcastMeta helper; doc comment + tests
updated (incl. an empty-list-with-metadata case).

* feat(output): recognise real DingTalk envelope keys in -f csv/ndjson/table

之前 -f csv 和 -f ndjson 的 list 检测白名单只认 items/results/data/list/
records/tools/servers/products,但真实钉钉响应用的是 result(单数直接数组
或一层包裹)/documents/emailAccounts/todoCards/events/messages,导致大
部分 list 命令的 csv 输出退化成 key,value 二列、ndjson 退化成整包一行。

修复点:

1. preferredListKeys 扩展加上真实 envelope key(result/documents/
   emailAccounts/todoCards/events/messages),并把它升级为 csv/table/
   ndjson 共享的"单一事实源"——filter.go 的 findDataList 不再维护自己
   的本地副本,直接复用这个列表。
2. extractRowsFromMap 改为委托 findDataList,自动获得"一层深度"的
   wrapper 支持({result: {todoCards: [...]}} 这种 envelope 现在能识
   别)。meta 合并:outer + inner 双层 sibling 拉通,outer 同名 key
   优先(避免 inner 把外层 success/total 等覆盖掉)。
3. writeTableish 和 writeCSV 调整 unwrapPrimaryObject 和
   extractRowsFromMap 的优先级——先试 list 检测,没命中再走 unwrap,
   避免 {result: {todoCards: [...]}} 被 unwrap 剥掉外层后直接走
   key,value 分支。
4. findDataList 允许"空数组+preferred key" 命中,保留原来"空 list +
   metadata 仍渲染为表格 + meta 广播一行"的行为。

新增 TestTabularDetectsRealDingTalkEnvelopes:四个真实 envelope 形态
(contact user search 的 result 直接数组、doc search 的 documents 顶层、
mail mailbox list 的 emailAccounts、todo task list 的 result.todoCards
一层深度)验证 ndjson 行数和 csv 表头都符合预期。

真接口回归(已登录态跑过):
- dws contact user search -f csv  → name/userId 列正常出表
- dws todo task list -f ndjson    → 20 行一条任务,可 jq -r .subject 直接管
- dws doc search -f csv           → 10 行 + nextPageToken 等 meta 广播尾列
- dws schema -f csv               → 无回归

Closes part of #252 follow-up.
2026-05-11 22:15:59 +08:00

159 lines
5.3 KiB
Go

// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package output
import (
"encoding/csv"
"io"
)
// writeCSV renders a payload as RFC-4180 CSV.
//
// It mirrors the shape decisions `-f table` already makes (same helpers:
// normalizePayload / unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice /
// formatValue), so column order and value flattening stay consistent between
// the two formats:
//
// - a list of objects — either a bare [{...},...] or wrapped under a
// well-known key ({items|results|data|records|...}) — becomes a header row
// plus one row per element. The union of keys (sorted) is the column set;
// missing values are empty cells; nested objects/arrays render as compact
// JSON in the cell. Any sibling metadata of the list (total, hasMore, ...)
// is broadcast as extra trailing columns, repeated on every row, so a CSV
// consumer never loses it (CSV has no "two tables in one file" concept the
// way the table renderer's footer does). Meta keys that collide with a data
// column are skipped. An empty list still emits the header plus one row of
// empty data cells carrying just the meta values.
// - a single object becomes a two-column `key,value` CSV.
// - a non-uniform list or a scalar becomes a single-column `value` CSV.
//
// `--fields` projection composes for free: WriteFiltered applies SelectFields
// before Write reaches us, so the rows are already narrowed.
//
// encoding/csv.Writer handles quoting/escaping of commas, double quotes and
// embedded newlines; cell text goes through formatValue (which also strips
// terminal control sequences, same as the table renderer).
func writeCSV(w io.Writer, payload any) error {
normalized, err := normalizePayload(payload)
if err != nil {
return err
}
cw := csv.NewWriter(w)
switch typed := normalized.(type) {
case map[string]any:
// Try table extraction first so wrappers around list payloads
// (e.g. {result: {todoCards: [...]}}) render as a real table
// instead of being peeled by unwrapPrimaryObject and degraded
// to key/value rows. unwrapPrimaryObject is then the fallback
// for single-object wrappers like {invocation: {...}}.
if headers, rows, meta, ok := extractRowsFromMap(typed); ok {
headers, rows = broadcastMeta(headers, rows, meta)
return writeTableCSV(cw, headers, rows)
}
if inner, ok := unwrapPrimaryObject(typed); ok {
return writeKeyValueCSV(cw, inner)
}
return writeKeyValueCSV(cw, typed)
case []any:
headers, rows, _ := rowsFromSlice(typed)
return writeTableCSV(cw, headers, rows)
case nil:
// Nothing to write — emit an empty document rather than erroring.
cw.Flush()
return cw.Error()
default:
// Scalar: a single-cell, single-row CSV.
if err := cw.Write([]string{formatValue(normalized)}); err != nil {
return err
}
cw.Flush()
return cw.Error()
}
}
func writeTableCSV(cw *csv.Writer, headers []string, rows [][]string) error {
if err := cw.Write(headers); err != nil {
return err
}
for _, row := range rows {
// rowsFromSlice / extractRowsFromMap already guarantee
// len(row) == len(headers), but stay defensive against future callers.
if len(row) != len(headers) {
padded := make([]string, len(headers))
copy(padded, row)
row = padded
}
if err := cw.Write(row); err != nil {
return err
}
}
cw.Flush()
return cw.Error()
}
// broadcastMeta appends the list's sibling metadata (total, hasMore, ...) as
// trailing columns repeated on every row. Meta keys that collide with an
// existing data column are skipped. If there are no rows but there is meta, a
// single row of empty data cells is emitted so the meta values aren't lost.
func broadcastMeta(headers []string, rows [][]string, meta map[string]any) ([]string, [][]string) {
if len(meta) == 0 {
return headers, rows
}
existing := make(map[string]bool, len(headers))
for _, h := range headers {
existing[h] = true
}
var metaKeys []string
var metaVals []string
for _, k := range sortedMapKeys(meta) {
if existing[k] {
continue
}
metaKeys = append(metaKeys, k)
metaVals = append(metaVals, formatValue(meta[k]))
}
if len(metaKeys) == 0 {
return headers, rows
}
outHeaders := append(append([]string{}, headers...), metaKeys...)
if len(rows) == 0 {
emptyData := make([]string, len(headers))
return outHeaders, [][]string{append(emptyData, metaVals...)}
}
outRows := make([][]string, len(rows))
for i, r := range rows {
nr := make([]string, 0, len(outHeaders))
nr = append(nr, r...)
nr = append(nr, metaVals...)
outRows[i] = nr
}
return outHeaders, outRows
}
func writeKeyValueCSV(cw *csv.Writer, m map[string]any) error {
if err := cw.Write([]string{"key", "value"}); err != nil {
return err
}
for _, key := range sortedMapKeys(m) {
if err := cw.Write([]string{key, formatValue(m[key])}); err != nil {
return err
}
}
cw.Flush()
return cw.Error()
}