Files
dingtalk-workspace-cli/internal/output/ndjson_csv_test.go
T
修雨 c4952d0207 feat(output): add -f ndjson and -f csv (#252) (#259)
* feat(output): add -f ndjson (implemented) and scaffold -f csv (#252)

`larksuite/cli` exposes --format ndjson / csv; dws only had json/table/raw/
pretty. This adds both as recognised global formats:

- ndjson: fully implemented (internal/output/ndjson.go) — top-level arrays
  and well-known list wrappers ({items|results|data|records|...}) emit one
  compact JSON document per line; anything else degrades to a single line.
  Streaming-friendly counterpart to `-f json`.
- csv: scaffolded (internal/output/csv.go) — writeCSV currently returns a
  clear "not implemented (#252)" error rather than silently degrading to
  JSON. A TODO block spells out the planned implementation (reuse the table
  renderer's extractRowsFromMap/rowsFromSlice flattening + encoding/csv).

Wired through Write() and normalizeFormat(); --format help string now lists
ndjson (csv to be added when writeCSV lands). Tests cover ndjson rendering
(array / wrapped-list / scalar) and pin the csv not-implemented contract.

Skeleton PR for #252 — ndjson is shippable as-is; csv is left for a
follow-up commit on this branch.

* feat(output): implement -f csv (#252)

Completes the CSV half of the format work. writeCSV mirrors the shape
decisions `-f table` already makes (reuses normalizePayload /
unwrapPrimaryObject / extractRowsFromMap / rowsFromSlice / formatValue) so
columns and value flattening are consistent between the two formats:

- a list of objects (bare or wrapped under items/results/data/records/...) →
  header row + one row per element; union of keys sorted; missing values are
  empty cells; nested objects/arrays render as compact JSON in the cell;
  sibling metadata of the list (total, hasMore, ...) is dropped.
- a single object → two-column key,value CSV (keys sorted).
- a non-uniform list / scalar → single-column `value` CSV.
- empty / nil → empty document.

encoding/csv.Writer handles RFC-4180 quoting (commas, quotes, newlines);
cells go through formatValue (also strips terminal control sequences, same
as the table renderer). `--fields` projection composes for free since
WriteFiltered applies SelectFields before Write.

--format help now lists csv; FormatCSV doc comment dropped the WIP marker;
the not-implemented test is replaced with real coverage (list with
comma/CJK/nested-array-as-JSON, wrapped-list-with-metadata, single object,
scalar) plus a --fields composition test.

* feat(output): broadcast list metadata as trailing columns in -f csv (#252)

Per review preference: instead of dropping the list's sibling metadata
(total, hasMore, ...) when rendering {records:[...], total:N} as CSV, append
each meta key as a trailing column repeated on every row, so a CSV consumer
never silently loses it (CSV has no "footer table" the way the table
renderer does). Meta keys colliding with a data column are skipped; an empty
list still emits the header (data + meta) plus one row of empty data cells
carrying the meta values. New broadcastMeta helper; doc comment + tests
updated (incl. an empty-list-with-metadata case).

* feat(output): recognise real DingTalk envelope keys in -f csv/ndjson/table

之前 -f csv 和 -f ndjson 的 list 检测白名单只认 items/results/data/list/
records/tools/servers/products,但真实钉钉响应用的是 result(单数直接数组
或一层包裹)/documents/emailAccounts/todoCards/events/messages,导致大
部分 list 命令的 csv 输出退化成 key,value 二列、ndjson 退化成整包一行。

修复点:

1. preferredListKeys 扩展加上真实 envelope key(result/documents/
   emailAccounts/todoCards/events/messages),并把它升级为 csv/table/
   ndjson 共享的"单一事实源"——filter.go 的 findDataList 不再维护自己
   的本地副本,直接复用这个列表。
2. extractRowsFromMap 改为委托 findDataList,自动获得"一层深度"的
   wrapper 支持({result: {todoCards: [...]}} 这种 envelope 现在能识
   别)。meta 合并:outer + inner 双层 sibling 拉通,outer 同名 key
   优先(避免 inner 把外层 success/total 等覆盖掉)。
3. writeTableish 和 writeCSV 调整 unwrapPrimaryObject 和
   extractRowsFromMap 的优先级——先试 list 检测,没命中再走 unwrap,
   避免 {result: {todoCards: [...]}} 被 unwrap 剥掉外层后直接走
   key,value 分支。
4. findDataList 允许"空数组+preferred key" 命中,保留原来"空 list +
   metadata 仍渲染为表格 + meta 广播一行"的行为。

新增 TestTabularDetectsRealDingTalkEnvelopes:四个真实 envelope 形态
(contact user search 的 result 直接数组、doc search 的 documents 顶层、
mail mailbox list 的 emailAccounts、todo task list 的 result.todoCards
一层深度)验证 ndjson 行数和 csv 表头都符合预期。

真接口回归(已登录态跑过):
- dws contact user search -f csv  → name/userId 列正常出表
- dws todo task list -f ndjson    → 20 行一条任务,可 jq -r .subject 直接管
- dws doc search -f csv           → 10 行 + nextPageToken 等 meta 广播尾列
- dws schema -f csv               → 无回归

Closes part of #252 follow-up.
2026-05-11 22:15:59 +08:00

236 lines
7.3 KiB
Go

// Copyright 2026 Alibaba Group
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package output
import (
"bytes"
"strings"
"testing"
)
func TestNormalizeFormatRecognizesNDJSONAndCSV(t *testing.T) {
if got := normalizeFormat("ndjson", FormatJSON); got != FormatNDJSON {
t.Errorf("normalizeFormat(ndjson) = %q, want %q", got, FormatNDJSON)
}
if got := normalizeFormat("CSV", FormatJSON); got != FormatCSV {
t.Errorf("normalizeFormat(CSV) = %q, want %q", got, FormatCSV)
}
}
func TestWriteNDJSON(t *testing.T) {
cases := []struct {
name string
payload any
wantLines []string
}{
{
name: "top-level array",
payload: []any{map[string]any{"id": "1"}, map[string]any{"id": "2"}},
wantLines: []string{`{"id":"1"}`, `{"id":"2"}`},
},
{
name: "wrapped list",
payload: map[string]any{"items": []any{map[string]any{"id": "1"}, map[string]any{"id": "2"}}, "count": 2},
wantLines: []string{`{"id":"1"}`, `{"id":"2"}`},
},
{
name: "scalar-ish object",
payload: map[string]any{"ok": true},
wantLines: []string{`{"ok":true}`},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
var buf bytes.Buffer
if err := Write(&buf, FormatNDJSON, tc.payload); err != nil {
t.Fatalf("Write(ndjson) error = %v", err)
}
got := strings.Split(strings.TrimRight(buf.String(), "\n"), "\n")
if len(got) != len(tc.wantLines) {
t.Fatalf("got %d lines %q, want %d %q", len(got), got, len(tc.wantLines), tc.wantLines)
}
for i, want := range tc.wantLines {
if strings.TrimSpace(got[i]) != want {
t.Errorf("line %d = %q, want %q", i, got[i], want)
}
}
})
}
}
func TestWriteCSV(t *testing.T) {
cases := []struct {
name string
payload any
want string
}{
{
// Union of keys (sorted), missing values → empty cells, a field with
// a comma gets quoted, CJK passes through verbatim, a nested array is
// rendered as compact JSON with its quotes CSV-escaped.
name: "list of objects",
payload: []any{
map[string]any{"id": "1", "name": "张三"},
map[string]any{"id": "2", "name": "Bob, Jr."},
map[string]any{"id": "3", "tags": []any{"x", "y"}},
},
want: "id,name,tags\n" +
"1,张三,\n" +
"2,\"Bob, Jr.\",\n" +
"3,,\"[\"\"x\"\",\"\"y\"\"]\"\n",
},
{
// {records:[...], total:N}: the list becomes the table; sibling
// metadata (total) is broadcast as a trailing column on every row.
name: "wrapped list with metadata",
payload: map[string]any{
"records": []any{map[string]any{"id": "1"}, map[string]any{"id": "2"}},
"total": 2,
},
want: "id,total\n1,2\n2,2\n",
},
{
// Empty list + metadata: still emit the header (data + meta) plus a
// single row of empty data cells carrying the meta values.
name: "empty wrapped list with metadata",
payload: map[string]any{
"records": []any{},
"total": 0,
"hasMore": false,
},
want: "value,hasMore,total\n,false,0\n",
},
{
// A plain object → two-column key,value CSV with keys sorted.
name: "single object",
payload: map[string]any{"ok": true, "name": "x"},
want: "key,value\nname,x\nok,true\n",
},
{
name: "scalar",
payload: "hello",
want: "hello\n",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
var buf bytes.Buffer
if err := Write(&buf, FormatCSV, tc.payload); err != nil {
t.Fatalf("Write(csv) error = %v", err)
}
if got := buf.String(); got != tc.want {
t.Errorf("Write(csv) =\n%q\nwant\n%q", got, tc.want)
}
})
}
}
// TestWriteCSVComposesWithFields guards that --fields projection (applied by
// WriteFiltered before Write) narrows the CSV columns.
func TestWriteCSVComposesWithFields(t *testing.T) {
payload := map[string]any{
"items": []any{
map[string]any{"id": "1", "name": "Alice", "secret": "s1"},
map[string]any{"id": "2", "name": "Bob", "secret": "s2"},
},
}
var buf bytes.Buffer
if err := WriteFiltered(&buf, FormatCSV, payload, "id,name", ""); err != nil {
t.Fatalf("WriteFiltered(csv) error = %v", err)
}
got := buf.String()
if strings.Contains(got, "secret") || strings.Contains(got, "s1") {
t.Errorf("--fields did not drop the secret column; got:\n%s", got)
}
if !strings.Contains(got, "id,name") || !strings.Contains(got, "Alice") {
t.Errorf("expected projected columns id,name with values; got:\n%s", got)
}
}
// TestTabularDetectsRealDingTalkEnvelopes guards against shipping a -f csv /
// -f ndjson that degrades to one-line-key-value for the envelope shapes the
// real product surface actually returns. Each case is a payload shape observed
// in production (contact / doc / mail / todo / chat search responses).
func TestTabularDetectsRealDingTalkEnvelopes(t *testing.T) {
cases := []struct {
name string
payload map[string]any
wantNDLines int // expected line count from -f ndjson
wantCSVHead string // first header line of -f csv
}{
{
name: "result direct array (contact user search)",
payload: map[string]any{
"result": []any{map[string]any{"name": "张三", "userId": "123"}, map[string]any{"name": "李四", "userId": "456"}},
"success": true,
},
wantNDLines: 2,
wantCSVHead: "name,userId,success",
},
{
name: "documents top-level (doc search)",
payload: map[string]any{
"documents": []any{map[string]any{"nodeId": "n1", "name": "A"}, map[string]any{"nodeId": "n2", "name": "B"}},
"hasMore": true,
"nextPageToken": "tok",
},
wantNDLines: 2,
wantCSVHead: "name,nodeId,hasMore,nextPageToken",
},
{
name: "emailAccounts top-level (mail mailbox list)",
payload: map[string]any{
"emailAccounts": []any{map[string]any{"email": "a@b.com", "type": "ORG"}},
"success": "true",
},
wantNDLines: 1,
wantCSVHead: "email,type,success",
},
{
name: "todoCards under result wrapper (todo task list)",
payload: map[string]any{
"result": map[string]any{
"todoCards": []any{
map[string]any{"taskId": "t1", "subject": "做一做"},
map[string]any{"taskId": "t2", "subject": "再做一做"},
},
},
},
wantNDLines: 2,
wantCSVHead: "subject,taskId",
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
var nd bytes.Buffer
if err := Write(&nd, FormatNDJSON, tc.payload); err != nil {
t.Fatalf("ndjson write: %v", err)
}
ndLines := strings.Split(strings.TrimRight(nd.String(), "\n"), "\n")
if len(ndLines) != tc.wantNDLines {
t.Errorf("ndjson: got %d lines %q, want %d", len(ndLines), ndLines, tc.wantNDLines)
}
var c bytes.Buffer
if err := Write(&c, FormatCSV, tc.payload); err != nil {
t.Fatalf("csv write: %v", err)
}
gotHead := strings.SplitN(c.String(), "\n", 2)[0]
if gotHead != tc.wantCSVHead {
t.Errorf("csv header: got %q, want %q", gotHead, tc.wantCSVHead)
}
})
}
}