* feat(output): add -f ndjson (implemented) and scaffold -f csv (#252) `larksuite/cli` exposes --format ndjson / csv; dws only had json/table/raw/ pretty. This adds both as recognised global formats: - ndjson: fully implemented (internal/output/ndjson.go) — top-level arrays and well-known list wrappers ({items|results|data|records|...}) emit one compact JSON document per line; anything else degrades to a single line. Streaming-friendly counterpart to `-f json`. - csv: scaffolded (internal/output/csv.go) — writeCSV currently returns a clear "not implemented (#252)" error rather than silently degrading to JSON. A TODO block spells out the planned implementation (reuse the table renderer's extractRowsFromMap/rowsFromSlice flattening + encoding/csv). Wired through Write() and normalizeFormat(); --format help string now lists ndjson (csv to be added when writeCSV lands). Tests cover ndjson rendering (array / wrapped-list / scalar) and pin the csv not-implemented contract. Skeleton PR for #252 — ndjson is shippable as-is; csv is left for a follow-up commit on this branch. * feat(output): implement -f csv (#252) Completes the CSV half of the format work. writeCSV mirrors the shape decisions `-f table` already makes (reuses normalizePayload / unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice / formatValue) so columns and value flattening are consistent between the two formats: - a list of objects (bare or wrapped under items/results/data/records/...) → header row + one row per element; union of keys sorted; missing values are empty cells; nested objects/arrays render as compact JSON in the cell; sibling metadata of the list (total, hasMore, ...) is dropped. - a single object → two-column key,value CSV (keys sorted). - a non-uniform list / scalar → single-column `value` CSV. - empty / nil → empty document. encoding/csv.Writer handles RFC-4180 quoting (commas, quotes, newlines); cells go through formatValue (also strips terminal control sequences, same as the table renderer). `--fields` projection composes for free since WriteFiltered applies SelectFields before Write. --format help now lists csv; FormatCSV doc comment dropped the WIP marker; the not-implemented test is replaced with real coverage (list with comma/CJK/nested-array-as-JSON, wrapped-list-with-metadata, single object, scalar) plus a --fields composition test. * feat(output): broadcast list metadata as trailing columns in -f csv (#252) Per review preference: instead of dropping the list's sibling metadata (total, hasMore, ...) when rendering {records:[...], total:N} as CSV, append each meta key as a trailing column repeated on every row, so a CSV consumer never silently loses it (CSV has no "footer table" the way the table renderer does). Meta keys colliding with a data column are skipped; an empty list still emits the header (data + meta) plus one row of empty data cells carrying the meta values. New broadcastMeta helper; doc comment + tests updated (incl. an empty-list-with-metadata case). * feat(output): recognise real DingTalk envelope keys in -f csv/ndjson/table 之前 -f csv 和 -f ndjson 的 list 检测白名单只认 items/results/data/list/ records/tools/servers/products,但真实钉钉响应用的是 result(单数直接数组 或一层包裹)/documents/emailAccounts/todoCards/events/messages,导致大 部分 list 命令的 csv 输出退化成 key,value 二列、ndjson 退化成整包一行。 修复点: 1. preferredListKeys 扩展加上真实 envelope key(result/documents/ emailAccounts/todoCards/events/messages),并把它升级为 csv/table/ ndjson 共享的"单一事实源"——filter.go 的 findDataList 不再维护自己 的本地副本,直接复用这个列表。 2. extractRowsFromMap 改为委托 findDataList,自动获得"一层深度"的 wrapper 支持({result: {todoCards: [...]}} 这种 envelope 现在能识 别)。meta 合并:outer + inner 双层 sibling 拉通,outer 同名 key 优先(避免 inner 把外层 success/total 等覆盖掉)。 3. writeTableish 和 writeCSV 调整 unwrapPrimaryObject 和 extractRowsFromMap 的优先级——先试 list 检测,没命中再走 unwrap, 避免 {result: {todoCards: [...]}} 被 unwrap 剥掉外层后直接走 key,value 分支。 4. findDataList 允许"空数组+preferred key" 命中,保留原来"空 list + metadata 仍渲染为表格 + meta 广播一行"的行为。 新增 TestTabularDetectsRealDingTalkEnvelopes:四个真实 envelope 形态 (contact user search 的 result 直接数组、doc search 的 documents 顶层、 mail mailbox list 的 emailAccounts、todo task list 的 result.todoCards 一层深度)验证 ndjson 行数和 csv 表头都符合预期。 真接口回归(已登录态跑过): - dws contact user search -f csv → name/userId 列正常出表 - dws todo task list -f ndjson → 20 行一条任务,可 jq -r .subject 直接管 - dws doc search -f csv → 10 行 + nextPageToken 等 meta 广播尾列 - dws schema -f csv → 无回归 Closes part of #252 follow-up.
236 lines
7.3 KiB
Go
236 lines
7.3 KiB
Go
// Copyright 2026 Alibaba Group
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package output
|
|
|
|
import (
|
|
"bytes"
|
|
"strings"
|
|
"testing"
|
|
)
|
|
|
|
func TestNormalizeFormatRecognizesNDJSONAndCSV(t *testing.T) {
|
|
if got := normalizeFormat("ndjson", FormatJSON); got != FormatNDJSON {
|
|
t.Errorf("normalizeFormat(ndjson) = %q, want %q", got, FormatNDJSON)
|
|
}
|
|
if got := normalizeFormat("CSV", FormatJSON); got != FormatCSV {
|
|
t.Errorf("normalizeFormat(CSV) = %q, want %q", got, FormatCSV)
|
|
}
|
|
}
|
|
|
|
func TestWriteNDJSON(t *testing.T) {
|
|
cases := []struct {
|
|
name string
|
|
payload any
|
|
wantLines []string
|
|
}{
|
|
{
|
|
name: "top-level array",
|
|
payload: []any{map[string]any{"id": "1"}, map[string]any{"id": "2"}},
|
|
wantLines: []string{`{"id":"1"}`, `{"id":"2"}`},
|
|
},
|
|
{
|
|
name: "wrapped list",
|
|
payload: map[string]any{"items": []any{map[string]any{"id": "1"}, map[string]any{"id": "2"}}, "count": 2},
|
|
wantLines: []string{`{"id":"1"}`, `{"id":"2"}`},
|
|
},
|
|
{
|
|
name: "scalar-ish object",
|
|
payload: map[string]any{"ok": true},
|
|
wantLines: []string{`{"ok":true}`},
|
|
},
|
|
}
|
|
for _, tc := range cases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
var buf bytes.Buffer
|
|
if err := Write(&buf, FormatNDJSON, tc.payload); err != nil {
|
|
t.Fatalf("Write(ndjson) error = %v", err)
|
|
}
|
|
got := strings.Split(strings.TrimRight(buf.String(), "\n"), "\n")
|
|
if len(got) != len(tc.wantLines) {
|
|
t.Fatalf("got %d lines %q, want %d %q", len(got), got, len(tc.wantLines), tc.wantLines)
|
|
}
|
|
for i, want := range tc.wantLines {
|
|
if strings.TrimSpace(got[i]) != want {
|
|
t.Errorf("line %d = %q, want %q", i, got[i], want)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
func TestWriteCSV(t *testing.T) {
|
|
cases := []struct {
|
|
name string
|
|
payload any
|
|
want string
|
|
}{
|
|
{
|
|
// Union of keys (sorted), missing values → empty cells, a field with
|
|
// a comma gets quoted, CJK passes through verbatim, a nested array is
|
|
// rendered as compact JSON with its quotes CSV-escaped.
|
|
name: "list of objects",
|
|
payload: []any{
|
|
map[string]any{"id": "1", "name": "张三"},
|
|
map[string]any{"id": "2", "name": "Bob, Jr."},
|
|
map[string]any{"id": "3", "tags": []any{"x", "y"}},
|
|
},
|
|
want: "id,name,tags\n" +
|
|
"1,张三,\n" +
|
|
"2,\"Bob, Jr.\",\n" +
|
|
"3,,\"[\"\"x\"\",\"\"y\"\"]\"\n",
|
|
},
|
|
{
|
|
// {records:[...], total:N}: the list becomes the table; sibling
|
|
// metadata (total) is broadcast as a trailing column on every row.
|
|
name: "wrapped list with metadata",
|
|
payload: map[string]any{
|
|
"records": []any{map[string]any{"id": "1"}, map[string]any{"id": "2"}},
|
|
"total": 2,
|
|
},
|
|
want: "id,total\n1,2\n2,2\n",
|
|
},
|
|
{
|
|
// Empty list + metadata: still emit the header (data + meta) plus a
|
|
// single row of empty data cells carrying the meta values.
|
|
name: "empty wrapped list with metadata",
|
|
payload: map[string]any{
|
|
"records": []any{},
|
|
"total": 0,
|
|
"hasMore": false,
|
|
},
|
|
want: "value,hasMore,total\n,false,0\n",
|
|
},
|
|
{
|
|
// A plain object → two-column key,value CSV with keys sorted.
|
|
name: "single object",
|
|
payload: map[string]any{"ok": true, "name": "x"},
|
|
want: "key,value\nname,x\nok,true\n",
|
|
},
|
|
{
|
|
name: "scalar",
|
|
payload: "hello",
|
|
want: "hello\n",
|
|
},
|
|
}
|
|
for _, tc := range cases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
var buf bytes.Buffer
|
|
if err := Write(&buf, FormatCSV, tc.payload); err != nil {
|
|
t.Fatalf("Write(csv) error = %v", err)
|
|
}
|
|
if got := buf.String(); got != tc.want {
|
|
t.Errorf("Write(csv) =\n%q\nwant\n%q", got, tc.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// TestWriteCSVComposesWithFields guards that --fields projection (applied by
|
|
// WriteFiltered before Write) narrows the CSV columns.
|
|
func TestWriteCSVComposesWithFields(t *testing.T) {
|
|
payload := map[string]any{
|
|
"items": []any{
|
|
map[string]any{"id": "1", "name": "Alice", "secret": "s1"},
|
|
map[string]any{"id": "2", "name": "Bob", "secret": "s2"},
|
|
},
|
|
}
|
|
var buf bytes.Buffer
|
|
if err := WriteFiltered(&buf, FormatCSV, payload, "id,name", ""); err != nil {
|
|
t.Fatalf("WriteFiltered(csv) error = %v", err)
|
|
}
|
|
got := buf.String()
|
|
if strings.Contains(got, "secret") || strings.Contains(got, "s1") {
|
|
t.Errorf("--fields did not drop the secret column; got:\n%s", got)
|
|
}
|
|
if !strings.Contains(got, "id,name") || !strings.Contains(got, "Alice") {
|
|
t.Errorf("expected projected columns id,name with values; got:\n%s", got)
|
|
}
|
|
}
|
|
|
|
// TestTabularDetectsRealDingTalkEnvelopes guards against shipping a -f csv /
|
|
// -f ndjson that degrades to one-line-key-value for the envelope shapes the
|
|
// real product surface actually returns. Each case is a payload shape observed
|
|
// in production (contact / doc / mail / todo / chat search responses).
|
|
func TestTabularDetectsRealDingTalkEnvelopes(t *testing.T) {
|
|
cases := []struct {
|
|
name string
|
|
payload map[string]any
|
|
wantNDLines int // expected line count from -f ndjson
|
|
wantCSVHead string // first header line of -f csv
|
|
}{
|
|
{
|
|
name: "result direct array (contact user search)",
|
|
payload: map[string]any{
|
|
"result": []any{map[string]any{"name": "张三", "userId": "123"}, map[string]any{"name": "李四", "userId": "456"}},
|
|
"success": true,
|
|
},
|
|
wantNDLines: 2,
|
|
wantCSVHead: "name,userId,success",
|
|
},
|
|
{
|
|
name: "documents top-level (doc search)",
|
|
payload: map[string]any{
|
|
"documents": []any{map[string]any{"nodeId": "n1", "name": "A"}, map[string]any{"nodeId": "n2", "name": "B"}},
|
|
"hasMore": true,
|
|
"nextPageToken": "tok",
|
|
},
|
|
wantNDLines: 2,
|
|
wantCSVHead: "name,nodeId,hasMore,nextPageToken",
|
|
},
|
|
{
|
|
name: "emailAccounts top-level (mail mailbox list)",
|
|
payload: map[string]any{
|
|
"emailAccounts": []any{map[string]any{"email": "a@b.com", "type": "ORG"}},
|
|
"success": "true",
|
|
},
|
|
wantNDLines: 1,
|
|
wantCSVHead: "email,type,success",
|
|
},
|
|
{
|
|
name: "todoCards under result wrapper (todo task list)",
|
|
payload: map[string]any{
|
|
"result": map[string]any{
|
|
"todoCards": []any{
|
|
map[string]any{"taskId": "t1", "subject": "做一做"},
|
|
map[string]any{"taskId": "t2", "subject": "再做一做"},
|
|
},
|
|
},
|
|
},
|
|
wantNDLines: 2,
|
|
wantCSVHead: "subject,taskId",
|
|
},
|
|
}
|
|
for _, tc := range cases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
var nd bytes.Buffer
|
|
if err := Write(&nd, FormatNDJSON, tc.payload); err != nil {
|
|
t.Fatalf("ndjson write: %v", err)
|
|
}
|
|
ndLines := strings.Split(strings.TrimRight(nd.String(), "\n"), "\n")
|
|
if len(ndLines) != tc.wantNDLines {
|
|
t.Errorf("ndjson: got %d lines %q, want %d", len(ndLines), ndLines, tc.wantNDLines)
|
|
}
|
|
|
|
var c bytes.Buffer
|
|
if err := Write(&c, FormatCSV, tc.payload); err != nil {
|
|
t.Fatalf("csv write: %v", err)
|
|
}
|
|
gotHead := strings.SplitN(c.String(), "\n", 2)[0]
|
|
if gotHead != tc.wantCSVHead {
|
|
t.Errorf("csv header: got %q, want %q", gotHead, tc.wantCSVHead)
|
|
}
|
|
})
|
|
}
|
|
}
|