* feat(output): add -f ndjson (implemented) and scaffold -f csv (#252) `larksuite/cli` exposes --format ndjson / csv; dws only had json/table/raw/ pretty. This adds both as recognised global formats: - ndjson: fully implemented (internal/output/ndjson.go) — top-level arrays and well-known list wrappers ({items|results|data|records|...}) emit one compact JSON document per line; anything else degrades to a single line. Streaming-friendly counterpart to `-f json`. - csv: scaffolded (internal/output/csv.go) — writeCSV currently returns a clear "not implemented (#252)" error rather than silently degrading to JSON. A TODO block spells out the planned implementation (reuse the table renderer's extractRowsFromMap/rowsFromSlice flattening + encoding/csv). Wired through Write() and normalizeFormat(); --format help string now lists ndjson (csv to be added when writeCSV lands). Tests cover ndjson rendering (array / wrapped-list / scalar) and pin the csv not-implemented contract. Skeleton PR for #252 — ndjson is shippable as-is; csv is left for a follow-up commit on this branch. * feat(output): implement -f csv (#252) Completes the CSV half of the format work. writeCSV mirrors the shape decisions `-f table` already makes (reuses normalizePayload / unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice / formatValue) so columns and value flattening are consistent between the two formats: - a list of objects (bare or wrapped under items/results/data/records/...) → header row + one row per element; union of keys sorted; missing values are empty cells; nested objects/arrays render as compact JSON in the cell; sibling metadata of the list (total, hasMore, ...) is dropped. - a single object → two-column key,value CSV (keys sorted). - a non-uniform list / scalar → single-column `value` CSV. - empty / nil → empty document. encoding/csv.Writer handles RFC-4180 quoting (commas, quotes, newlines); cells go through formatValue (also strips terminal control sequences, same as the table renderer). `--fields` projection composes for free since WriteFiltered applies SelectFields before Write. --format help now lists csv; FormatCSV doc comment dropped the WIP marker; the not-implemented test is replaced with real coverage (list with comma/CJK/nested-array-as-JSON, wrapped-list-with-metadata, single object, scalar) plus a --fields composition test. * feat(output): broadcast list metadata as trailing columns in -f csv (#252) Per review preference: instead of dropping the list's sibling metadata (total, hasMore, ...) when rendering {records:[...], total:N} as CSV, append each meta key as a trailing column repeated on every row, so a CSV consumer never silently loses it (CSV has no "footer table" the way the table renderer does). Meta keys colliding with a data column are skipped; an empty list still emits the header (data + meta) plus one row of empty data cells carrying the meta values. New broadcastMeta helper; doc comment + tests updated (incl. an empty-list-with-metadata case). * feat(output): recognise real DingTalk envelope keys in -f csv/ndjson/table 之前 -f csv 和 -f ndjson 的 list 检测白名单只认 items/results/data/list/ records/tools/servers/products,但真实钉钉响应用的是 result(单数直接数组 或一层包裹)/documents/emailAccounts/todoCards/events/messages,导致大 部分 list 命令的 csv 输出退化成 key,value 二列、ndjson 退化成整包一行。 修复点: 1. preferredListKeys 扩展加上真实 envelope key(result/documents/ emailAccounts/todoCards/events/messages),并把它升级为 csv/table/ ndjson 共享的"单一事实源"——filter.go 的 findDataList 不再维护自己 的本地副本,直接复用这个列表。 2. extractRowsFromMap 改为委托 findDataList,自动获得"一层深度"的 wrapper 支持({result: {todoCards: [...]}} 这种 envelope 现在能识 别)。meta 合并:outer + inner 双层 sibling 拉通,outer 同名 key 优先(避免 inner 把外层 success/total 等覆盖掉)。 3. writeTableish 和 writeCSV 调整 unwrapPrimaryObject 和 extractRowsFromMap 的优先级——先试 list 检测,没命中再走 unwrap, 避免 {result: {todoCards: [...]}} 被 unwrap 剥掉外层后直接走 key,value 分支。 4. findDataList 允许"空数组+preferred key" 命中,保留原来"空 list + metadata 仍渲染为表格 + meta 广播一行"的行为。 新增 TestTabularDetectsRealDingTalkEnvelopes:四个真实 envelope 形态 (contact user search 的 result 直接数组、doc search 的 documents 顶层、 mail mailbox list 的 emailAccounts、todo task list 的 result.todoCards 一层深度)验证 ndjson 行数和 csv 表头都符合预期。 真接口回归(已登录态跑过): - dws contact user search -f csv → name/userId 列正常出表 - dws todo task list -f ndjson → 20 行一条任务,可 jq -r .subject 直接管 - dws doc search -f csv → 10 行 + nextPageToken 等 meta 广播尾列 - dws schema -f csv → 无回归 Closes part of #252 follow-up.
159 lines
5.3 KiB
Go
159 lines
5.3 KiB
Go
// Copyright 2026 Alibaba Group
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package output
|
|
|
|
import (
|
|
"encoding/csv"
|
|
"io"
|
|
)
|
|
|
|
// writeCSV renders a payload as RFC-4180 CSV.
|
|
//
|
|
// It mirrors the shape decisions `-f table` already makes (same helpers:
|
|
// normalizePayload / unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice /
|
|
// formatValue), so column order and value flattening stay consistent between
|
|
// the two formats:
|
|
//
|
|
// - a list of objects — either a bare [{...},...] or wrapped under a
|
|
// well-known key ({items|results|data|records|...}) — becomes a header row
|
|
// plus one row per element. The union of keys (sorted) is the column set;
|
|
// missing values are empty cells; nested objects/arrays render as compact
|
|
// JSON in the cell. Any sibling metadata of the list (total, hasMore, ...)
|
|
// is broadcast as extra trailing columns, repeated on every row, so a CSV
|
|
// consumer never loses it (CSV has no "two tables in one file" concept the
|
|
// way the table renderer's footer does). Meta keys that collide with a data
|
|
// column are skipped. An empty list still emits the header plus one row of
|
|
// empty data cells carrying just the meta values.
|
|
// - a single object becomes a two-column `key,value` CSV.
|
|
// - a non-uniform list or a scalar becomes a single-column `value` CSV.
|
|
//
|
|
// `--fields` projection composes for free: WriteFiltered applies SelectFields
|
|
// before Write reaches us, so the rows are already narrowed.
|
|
//
|
|
// encoding/csv.Writer handles quoting/escaping of commas, double quotes and
|
|
// embedded newlines; cell text goes through formatValue (which also strips
|
|
// terminal control sequences, same as the table renderer).
|
|
func writeCSV(w io.Writer, payload any) error {
|
|
normalized, err := normalizePayload(payload)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
cw := csv.NewWriter(w)
|
|
|
|
switch typed := normalized.(type) {
|
|
case map[string]any:
|
|
// Try table extraction first so wrappers around list payloads
|
|
// (e.g. {result: {todoCards: [...]}}) render as a real table
|
|
// instead of being peeled by unwrapPrimaryObject and degraded
|
|
// to key/value rows. unwrapPrimaryObject is then the fallback
|
|
// for single-object wrappers like {invocation: {...}}.
|
|
if headers, rows, meta, ok := extractRowsFromMap(typed); ok {
|
|
headers, rows = broadcastMeta(headers, rows, meta)
|
|
return writeTableCSV(cw, headers, rows)
|
|
}
|
|
if inner, ok := unwrapPrimaryObject(typed); ok {
|
|
return writeKeyValueCSV(cw, inner)
|
|
}
|
|
return writeKeyValueCSV(cw, typed)
|
|
case []any:
|
|
headers, rows, _ := rowsFromSlice(typed)
|
|
return writeTableCSV(cw, headers, rows)
|
|
case nil:
|
|
// Nothing to write — emit an empty document rather than erroring.
|
|
cw.Flush()
|
|
return cw.Error()
|
|
default:
|
|
// Scalar: a single-cell, single-row CSV.
|
|
if err := cw.Write([]string{formatValue(normalized)}); err != nil {
|
|
return err
|
|
}
|
|
cw.Flush()
|
|
return cw.Error()
|
|
}
|
|
}
|
|
|
|
func writeTableCSV(cw *csv.Writer, headers []string, rows [][]string) error {
|
|
if err := cw.Write(headers); err != nil {
|
|
return err
|
|
}
|
|
for _, row := range rows {
|
|
// rowsFromSlice / extractRowsFromMap already guarantee
|
|
// len(row) == len(headers), but stay defensive against future callers.
|
|
if len(row) != len(headers) {
|
|
padded := make([]string, len(headers))
|
|
copy(padded, row)
|
|
row = padded
|
|
}
|
|
if err := cw.Write(row); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
cw.Flush()
|
|
return cw.Error()
|
|
}
|
|
|
|
// broadcastMeta appends the list's sibling metadata (total, hasMore, ...) as
|
|
// trailing columns repeated on every row. Meta keys that collide with an
|
|
// existing data column are skipped. If there are no rows but there is meta, a
|
|
// single row of empty data cells is emitted so the meta values aren't lost.
|
|
func broadcastMeta(headers []string, rows [][]string, meta map[string]any) ([]string, [][]string) {
|
|
if len(meta) == 0 {
|
|
return headers, rows
|
|
}
|
|
existing := make(map[string]bool, len(headers))
|
|
for _, h := range headers {
|
|
existing[h] = true
|
|
}
|
|
var metaKeys []string
|
|
var metaVals []string
|
|
for _, k := range sortedMapKeys(meta) {
|
|
if existing[k] {
|
|
continue
|
|
}
|
|
metaKeys = append(metaKeys, k)
|
|
metaVals = append(metaVals, formatValue(meta[k]))
|
|
}
|
|
if len(metaKeys) == 0 {
|
|
return headers, rows
|
|
}
|
|
|
|
outHeaders := append(append([]string{}, headers...), metaKeys...)
|
|
if len(rows) == 0 {
|
|
emptyData := make([]string, len(headers))
|
|
return outHeaders, [][]string{append(emptyData, metaVals...)}
|
|
}
|
|
outRows := make([][]string, len(rows))
|
|
for i, r := range rows {
|
|
nr := make([]string, 0, len(outHeaders))
|
|
nr = append(nr, r...)
|
|
nr = append(nr, metaVals...)
|
|
outRows[i] = nr
|
|
}
|
|
return outHeaders, outRows
|
|
}
|
|
|
|
func writeKeyValueCSV(cw *csv.Writer, m map[string]any) error {
|
|
if err := cw.Write([]string{"key", "value"}); err != nil {
|
|
return err
|
|
}
|
|
for _, key := range sortedMapKeys(m) {
|
|
if err := cw.Write([]string{key, formatValue(m[key])}); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
cw.Flush()
|
|
return cw.Error()
|
|
}
|