Flatten

The "flatten" transform converts fields that hold arrays into distinct, individual data objects. This creates a new sequence of data, where each element encompasses both an extracted array component and all the original fields from the corresponding input object.

Parameters

as

The output field name(s) for the flattened field.

Default: the input fields.

fields

The field(s) to flatten. If no field is defined, the data object itself is treated as an array to be flattened.

index

The output field name for the zero-based index of the array values. If unspecified, an index field is not added.

Example

Single-Field Flattening

This example flattens the array-valued field named foo. Note that all fields except foo are repeated in every output datum.

{ "type": "flatten", "fields": ["foo"] }

Input data:

[
  { "name": "alpha", "data": 123, "foo": [1, 2] },
  { "name": "beta", "data": 456, "foo": [3, 4, 5] }
]

Result:

[
  { "name": "alpha", "data": 123, "foo": 1 },
  { "name": "alpha", "data": 123, "foo": 2 },
  { "name": "beta", "data": 456, "foo": 3 },
  { "name": "beta", "data": 456, "foo": 4 },
  { "name": "beta", "data": 456, "foo": 5 }
]

Adding an Index Field

{ "type": "flatten", "fields": ["foo"], "index": "idx" }

This example adds an field containing the array index that each item originated from.

[
  { "name": "alpha", "data": 123, "foo": [1, 2] },
  { "name": "beta", "data": 456, "foo": [3, 4, 5] }
]

Result:

[
  { "name": "alpha", "data": 123, "foo": 1, "idx": 0 },
  { "name": "alpha", "data": 123, "foo": 2, "idx": 1 },
  { "name": "beta", "data": 456, "foo": 3, "idx": 0 },
  { "name": "beta", "data": 456, "foo": 4, "idx": 1 },
  { "name": "beta", "data": 456, "foo": 5, "idx": 2 }
]

Multi-Field Flattening

{ "type": "flatten", "fields": ["foo", "bar"] }

This example simultaneously flattens the array-valued fields foo and bar. Given the input data

[
  { "key": "alpha", "foo": [1, 2], "bar": ["A", "B"] },
  { "key": "beta", "foo": [3, 4, 5], "bar": ["C", "D"] }
]

this example produces the output:

[
  { "key": "alpha", "foo": 1, "bar": "A" },
  { "key": "alpha", "foo": 2, "bar": "B" },
  { "key": "beta", "foo": 3, "bar": "C" },
  { "key": "beta", "foo": 4, "bar": "D" },
  { "key": "beta", "foo": 5, "bar": null }
]

Flattening Array Objects

{ "type": "flatten" }

This example treats the data objects as arrays that should be flattened. Given the input data

[[{ "foo": 1 }], [{ "foo": 2 }, { "foo": 3 }]]

this example produces the output:

[{ "foo": 1 }, { "foo": 2 }, { "foo": 3 }]