* feat(output): add -f ndjson (implemented) and scaffold -f csv (#252) `larksuite/cli` exposes --format ndjson / csv; dws only had json/table/raw/ pretty. This adds both as recognised global formats: - ndjson: fully implemented (internal/output/ndjson.go) — top-level arrays and well-known list wrappers ({items|results|data|records|...}) emit one compact JSON document per line; anything else degrades to a single line. Streaming-friendly counterpart to `-f json`. - csv: scaffolded (internal/output/csv.go) — writeCSV currently returns a clear "not implemented (#252)" error rather than silently degrading to JSON. A TODO block spells out the planned implementation (reuse the table renderer's extractRowsFromMap/rowsFromSlice flattening + encoding/csv). Wired through Write() and normalizeFormat(); --format help string now lists ndjson (csv to be added when writeCSV lands). Tests cover ndjson rendering (array / wrapped-list / scalar) and pin the csv not-implemented contract. Skeleton PR for #252 — ndjson is shippable as-is; csv is left for a follow-up commit on this branch. * feat(output): implement -f csv (#252) Completes the CSV half of the format work. writeCSV mirrors the shape decisions `-f table` already makes (reuses normalizePayload / unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice / formatValue) so columns and value flattening are consistent between the two formats: - a list of objects (bare or wrapped under items/results/data/records/...) → header row + one row per element; union of keys sorted; missing values are empty cells; nested objects/arrays render as compact JSON in the cell; sibling metadata of the list (total, hasMore, ...) is dropped. - a single object → two-column key,value CSV (keys sorted). - a non-uniform list / scalar → single-column `value` CSV. - empty / nil → empty document. encoding/csv.Writer handles RFC-4180 quoting (commas, quotes, newlines); cells go through formatValue (also strips terminal control sequences, same as the table renderer). `--fields` projection composes for free since WriteFiltered applies SelectFields before Write. --format help now lists csv; FormatCSV doc comment dropped the WIP marker; the not-implemented test is replaced with real coverage (list with comma/CJK/nested-array-as-JSON, wrapped-list-with-metadata, single object, scalar) plus a --fields composition test. * feat(output): broadcast list metadata as trailing columns in -f csv (#252) Per review preference: instead of dropping the list's sibling metadata (total, hasMore, ...) when rendering {records:[...], total:N} as CSV, append each meta key as a trailing column repeated on every row, so a CSV consumer never silently loses it (CSV has no "footer table" the way the table renderer does). Meta keys colliding with a data column are skipped; an empty list still emits the header (data + meta) plus one row of empty data cells carrying the meta values. New broadcastMeta helper; doc comment + tests updated (incl. an empty-list-with-metadata case). * feat(output): recognise real DingTalk envelope keys in -f csv/ndjson/table 之前 -f csv 和 -f ndjson 的 list 检测白名单只认 items/results/data/list/ records/tools/servers/products,但真实钉钉响应用的是 result(单数直接数组 或一层包裹)/documents/emailAccounts/todoCards/events/messages,导致大 部分 list 命令的 csv 输出退化成 key,value 二列、ndjson 退化成整包一行。 修复点: 1. preferredListKeys 扩展加上真实 envelope key(result/documents/ emailAccounts/todoCards/events/messages),并把它升级为 csv/table/ ndjson 共享的"单一事实源"——filter.go 的 findDataList 不再维护自己 的本地副本,直接复用这个列表。 2. extractRowsFromMap 改为委托 findDataList,自动获得"一层深度"的 wrapper 支持({result: {todoCards: [...]}} 这种 envelope 现在能识 别)。meta 合并:outer + inner 双层 sibling 拉通,outer 同名 key 优先(避免 inner 把外层 success/total 等覆盖掉)。 3. writeTableish 和 writeCSV 调整 unwrapPrimaryObject 和 extractRowsFromMap 的优先级——先试 list 检测,没命中再走 unwrap, 避免 {result: {todoCards: [...]}} 被 unwrap 剥掉外层后直接走 key,value 分支。 4. findDataList 允许"空数组+preferred key" 命中,保留原来"空 list + metadata 仍渲染为表格 + meta 广播一行"的行为。 新增 TestTabularDetectsRealDingTalkEnvelopes:四个真实 envelope 形态 (contact user search 的 result 直接数组、doc search 的 documents 顶层、 mail mailbox list 的 emailAccounts、todo task list 的 result.todoCards 一层深度)验证 ndjson 行数和 csv 表头都符合预期。 真接口回归(已登录态跑过): - dws contact user search -f csv → name/userId 列正常出表 - dws todo task list -f ndjson → 20 行一条任务,可 jq -r .subject 直接管 - dws doc search -f csv → 10 行 + nextPageToken 等 meta 广播尾列 - dws schema -f csv → 无回归 Closes part of #252 follow-up.
87 lines
2.6 KiB
Go
87 lines
2.6 KiB
Go
// Copyright 2026 Alibaba Group
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package output
|
|
|
|
import (
|
|
"bufio"
|
|
"encoding/json"
|
|
"io"
|
|
)
|
|
|
|
// writeNDJSON renders payload as newline-delimited JSON (https://ndjson.org):
|
|
// - a top-level array → one element per line
|
|
// - an object that wraps a → one element of that list per line
|
|
// well-known list key (items / results / data / records / value / ...)
|
|
// - anything else → a single line containing the whole value
|
|
//
|
|
// This is the streaming-friendly counterpart to `-f json`: each line is an
|
|
// independent, compact JSON document so consumers can `jq -c`, `while read`,
|
|
// or pipe into log pipelines without buffering the whole response.
|
|
//
|
|
// TODO(#252): consider honouring --fields per-line projection here too (today
|
|
// WriteFiltered already applies SelectFields before Write is reached, so this
|
|
// works, but a dedicated test would be good). Also decide whether non-list
|
|
// payloads should error under `-f ndjson` instead of degrading to one line.
|
|
func writeNDJSON(w io.Writer, payload any) error {
|
|
normalized, err := roundTripJSON(payload)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
bw := bufio.NewWriter(w)
|
|
enc := json.NewEncoder(bw)
|
|
// json.Encoder.Encode already appends a trailing newline per call.
|
|
|
|
switch v := normalized.(type) {
|
|
case []any:
|
|
for _, item := range v {
|
|
if err := enc.Encode(item); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
case map[string]any:
|
|
if loc := findDataList(v); loc != nil {
|
|
for _, item := range loc.list {
|
|
if err := enc.Encode(item); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
} else {
|
|
if err := enc.Encode(v); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
default:
|
|
if err := enc.Encode(v); err != nil {
|
|
return err
|
|
}
|
|
}
|
|
return bw.Flush()
|
|
}
|
|
|
|
// roundTripJSON normalizes an arbitrary Go value into the
|
|
// map[string]any / []any / scalar shape used by the rest of this package by
|
|
// marshalling and unmarshalling it through encoding/json.
|
|
func roundTripJSON(payload any) (any, error) {
|
|
raw, err := json.Marshal(payload)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
var out any
|
|
if err := json.Unmarshal(raw, &out); err != nil {
|
|
return nil, err
|
|
}
|
|
return out, nil
|
|
}
|