Flatten¶
The "flatten"
transform converts fields that hold arrays into distinct,
individual data objects. This creates a new sequence of data, where each
element encompasses both an extracted array component and all the original
fields from the corresponding input object.
Parameters¶
as
-
Type: string[] | string
The output field name(s) for the flattened field.
Default: the input fields.
fields
-
Type: string (field name)[] | string (field name)
The field(s) to flatten. If no field is defined, the data object itself is treated as an array to be flattened.
index
-
Type: string
The output field name for the zero-based index of the array values. If unspecified, an index field is not added.
Example¶
Single-Field Flattening¶
This example flattens the array-valued field named foo
. Note that all fields except foo
are repeated in every output datum.
{ "type": "flatten", "fields": ["foo"] }
Input data:
[
{ "name": "alpha", "data": 123, "foo": [1, 2] },
{ "name": "beta", "data": 456, "foo": [3, 4, 5] }
]
Result:
[
{ "name": "alpha", "data": 123, "foo": 1 },
{ "name": "alpha", "data": 123, "foo": 2 },
{ "name": "beta", "data": 456, "foo": 3 },
{ "name": "beta", "data": 456, "foo": 4 },
{ "name": "beta", "data": 456, "foo": 5 }
]
Adding an Index Field¶
{ "type": "flatten", "fields": ["foo"], "index": "idx" }
This example adds an field containing the array index that each item originated from.
[
{ "name": "alpha", "data": 123, "foo": [1, 2] },
{ "name": "beta", "data": 456, "foo": [3, 4, 5] }
]
Result:
[
{ "name": "alpha", "data": 123, "foo": 1, "idx": 0 },
{ "name": "alpha", "data": 123, "foo": 2, "idx": 1 },
{ "name": "beta", "data": 456, "foo": 3, "idx": 0 },
{ "name": "beta", "data": 456, "foo": 4, "idx": 1 },
{ "name": "beta", "data": 456, "foo": 5, "idx": 2 }
]
Multi-Field Flattening¶
{ "type": "flatten", "fields": ["foo", "bar"] }
This example simultaneously flattens the array-valued fields foo
and bar
. Given the input data
[
{ "key": "alpha", "foo": [1, 2], "bar": ["A", "B"] },
{ "key": "beta", "foo": [3, 4, 5], "bar": ["C", "D"] }
]
this example produces the output:
[
{ "key": "alpha", "foo": 1, "bar": "A" },
{ "key": "alpha", "foo": 2, "bar": "B" },
{ "key": "beta", "foo": 3, "bar": "C" },
{ "key": "beta", "foo": 4, "bar": "D" },
{ "key": "beta", "foo": 5, "bar": null }
]
Flattening Array Objects¶
{ "type": "flatten" }
This example treats the data objects as arrays that should be flattened. Given the input data
[[{ "foo": 1 }], [{ "foo": 2 }, { "foo": 3 }]]
this example produces the output:
[{ "foo": 1 }, { "foo": 2 }, { "foo": 3 }]