Transforms¶

A transform changes the rows that flow from a chart’s data source to its marks. Transforms can remove rows, add fields, summarize groups, or perform more specialized visualization operations. They run in the browser before the marks are drawn.

Transforms leave source data alone

Transforms change the rows used by a chart. They do not modify the original Python data.

Filter rows¶

A filter keeps rows for which its predicate is true:

import genome_spy as gs

measurements = [
    {"sample": "C1", "group": "control", "response": 0.42, "quality": 0.92},
    {"sample": "C2", "group": "control", "response": 0.55, "quality": 0.64},
    {"sample": "C3", "group": "control", "response": 0.61, "quality": 0.81},
    {"sample": "T1", "group": "treated", "response": 0.48, "quality": 0.58},
    {"sample": "T2", "group": "treated", "response": 0.72, "quality": 0.88},
    {"sample": "T3", "group": "treated", "response": 0.84, "quality": 0.95},
]

filtered_chart = (
    gs.Chart(measurements)
    .transform_filter(gs.datum.quality >= 0.7)
    .mark_point(filled=True, size=100)
    .encode(
        x=gs.X("response:Q").scale(domain=[0, 1]).title("Response"),
        y=gs.Y("sample:N").title("Passing sample"),
        color=gs.Color("group:N").legend(title="Group"),
        tooltip=["sample:N", "quality:Q"],
    )
    .properties(title="Keep rows with sufficient quality")
)

datum means “the row currently being considered.” In the example, datum.quality is at least 0.7, so four rows remain. Use the following Python syntax to build short conditions:

Write	Meaning
`datum`.quality	Read the `quality` value from the current row.
`>=`, `<`, `==`	Compare values.
`&`, `\|`	Both conditions, or either condition.
`~`	Reverse a condition.

For a small calculation inside a condition, use the expr helpers, such as if_() or isValid().

The GenomeSpy documentation describes the expression language and the constants and functions available inside an expression, as well as the filter transform itself.

Derive a field with a formula¶

A formula calculates a value and stores it in a new field. transform_calculate() accepts the output field as a keyword:

formula_chart = (
    gs.Chart(measurements)
    .transform_calculate(responsePercent=gs.datum.response * 100)
    .mark_point(filled=True, size=100)
    .encode(
        x=gs.X("sample:N").title("Sample"),
        y=gs.Y("responsePercent:Q").scale(domain=[0, 100]).title("Response (%)"),
        color=gs.Color("group:N").legend(title="Group"),
    )
    .properties(title="Derive a percentage field")
)

For the first row, the formula adds responsePercent=42. Existing fields such as sample, group, and response remain available. Encodings and later transforms can refer to the derived field by name.

Formula transforms are useful for small visualization-specific calculations: converting units, constructing labels, calculating interval endpoints, or deriving a category used only by the chart. See the formula transform in the GenomeSpy documentation.

Summarize groups¶

An aggregate transform reduces many rows into summary rows. transform_aggregate() uses groupby to choose which input rows belong together; fields, ops, and as_ are parallel lists that specify the input fields, aggregate operations, and output names:

aggregate_chart = (
    gs.Chart(measurements)
    .transform_calculate(
        as_="responsePercent",
        calculate=gs.datum.response * 100,
    )
    .transform_aggregate(
        groupby=["group"],
        fields=["responsePercent"],
        ops=["mean"],
        as_=["meanResponse"],
    )
    .mark_point(filled=True, size=140)
    .encode(
        x=gs.X("group:N").title("Group"),
        y=gs.Y("meanResponse:Q").scale(domain=[0, 100]).title("Mean response (%)"),
        color=gs.Color("group:N").legend(None),
        tooltip=[gs.Tooltip("group:N"), gs.Tooltip("meanResponse:Q").format(".1f")],
    )
    .properties(title="One summarized row per group")
)

The six input rows become two output rows, one for control and one for treated. Available operations include count, sum, min, max, mean, median, quartiles, and variance. The aggregate transform lists every supported operation.

Convenient Python forms¶

These shortcuts make common transforms easier to read. They behave like their GenomeSpy counterparts:

Method	Use it when you want to…
`transform_calculate()`	add one or more calculated columns.
`transform_flatten()`	turn values nested in a list into separate rows.
`transform_sample()`	draw a smaller sample of a large table.

For example, calculations can use output keywords:

chart = chart.transform_calculate(
    doubled=gs.datum.value * 2,
    centered=gs.datum.value - 10,
)

Multiple keyword calculations are added in their written order. Use the direct form when the output name is only known dynamically:

chart = chart.transform_calculate(
    as_=output_name,
    calculate=gs.datum.response * 100,
)

The Chart API reference lists every available transform method.

Transform order matters¶

Transforms run from top to bottom, and each step receives the output of the previous step. In the aggregate example:

transform_calculate() adds responsePercent to every row.
transform_aggregate() groups those rows and reads the new field.
Encodings read group and meanResponse from the two summary rows.

Reversing the first two transforms would fail because responsePercent would not exist when the aggregate tried to read it. The serialized transform list preserves the same order as the method chain.

Filtering before an aggregate changes which rows contribute to the summary; filtering afterward tests the summary rows instead. Choose the order from the question the visualization should answer.

Cache rows for interactive transforms¶

Place transform_collect() before a formula or filter that depends on an interactive parameter. It stores incoming rows in the browser so GenomeSpy can replay them when the parameter changes, avoiding a return to the data source’s loading path and a possible loading spinner. For example, with a slider parameter cutoff attached to the chart:

chart = chart.transform_collect().transform_filter(gs.datum.score >= cutoff)

Order matters: collect the rows before the parameter-dependent transform, so it can recalculate from those rows. You do not need a collector before every transform; filters that only read fixed data fields do not rerun in response to slider changes. Collecting retains rows in browser memory.

Transform in Python or in GenomeSpy?¶

Prepare data in Python when the operation:

cleans or validates source data;
performs statistical analysis;
joins or reshapes a large table;
produces a reusable result needed outside the visualization.

Use a GenomeSpy transform when the operation is part of the visual specification, should run after browser-side data loading, or needs to react to an interactive parameter. Keeping that boundary clear makes both the analysis and visualization easier to test.

The linked genome tracks example chains nine of them, including pileup, label placement, and coordinate linearization.

GenomeSpy also provides transforms for sorting and stacking, lookups, windows, genomic coordinates, sequence data, read alignments, and label placement. Those transforms are introduced by the focused examples that need them; their complete signatures are available in the API reference, and the GenomeSpy documentation describes each one in its transform reference.