Visualization Grammar

The original GenomeSpy was what is now called the GenomeSpy App: an interactive analytics application for exploring genomic data across sample collections, particularly cancer cohorts. Underneath it is GenomeSpy Core, a lower-level visualization package that provides a rendering engine, graphical marks, scales, data loaders and transformations, and other building blocks. Core and App were initially a single, bloated package, but nowadays they have clearly separated roles. Over the years, Core has grown into a general-purpose visualization toolkit that is useful for all sorts of genomic visualization tasks—and even some non-genomic ones.

Unlike a traditional genome browser, GenomeSpy does not provide a fixed collection of track types. Visualizations are instead composed using a declarative visualization grammar from lower-level building blocks. This enables much greater flexibility for highly customized visualizations and allows GenomeSpy to adapt easily to different datasets and applications. However, the cost is a somewhat steeper learning curve.

Most bioinformaticians are likely familiar with ggplot2, a grammar of graphics for R. GenomeSpy owes much of its grammar design to another, slightly less well-known one: Vega-Lite. While not everything in Vega-Lite fits genomic data cleanly, it got many things right. However, GenomeSpy’s internal architecture is quite different. I designed it for interaction with large amounts of non-aggregated data, which is particularly important in genomic exploration but less commonly needed in ordinary statistical plots.

Lingering in version 0.x

GenomeSpy has remained in v0.x for years. In semantic versioning, version zero means that basically anything can still change in a breaking manner. Still, GenomeSpy’s grammar and API have stayed remarkably stable, mostly because numerous existing visualizations depend on them.

I’ve nevertheless hesitated to release v1.0. There have always been annoying corner cases, essential features to be implemented, and documentation that has been insufficient. In addition, much of GenomeSpy’s feature set was defined by the needs of the research projects aimed at overcoming drug resistance in ovarian cancer rather than building polished software products. But the main reason, when I think about it now, has been my perfectionism, which isn’t always a useful trait.

I graduated some time ago and finally got an opportunity to focus on this unfinished business. During the last several months, I’ve fixed quite a lot of these issues. While I am proficient at writing code, coding agents, especially OpenAI Codex, have provided an immense productivity boost. Some annoying corner cases remain, but there always will be. I’m also not nearly done with all the ideas I’ve come up with over the years. But importantly, keeping the project at version zero because of them makes no sense.

I have now taken a bold step: GenomeSpy 1.0 is out.

What’s new in 1.0

There isn’t one single feature that defines this release. In fact, there’s nothing new in v1.0, as it is solely a version bump from the earlier v0.90.0. But not long ago, I wouldn’t have been comfortable with the bump.

Much of the recent work has been about making existing pieces more complete and predictable, something that a user would expect from a relatively mature visualization package. Some of that work is described briefly below.

Data support has expanded to formats including Parquet, BED, BEDPE, BAM, and compressed files, and several new transforms have been added for tasks such as coordinate lookup, window operations, set intersections, and label displacement. These building blocks enabled many new non-trivial examples shown in the documentation.

There has also been a lot of work on not-so-exciting features that are still important when creating plots and visualizations for others to view. Legends, for example. Axes and view titles now allocate their required space automatically, and clipping and layout behave better in dense compositions with scrollable viewports. Previously, users had to adjust these manually.

GenomeSpy uses a GPU-first design. While nearly all PCs, Macs, tablets, and phones have a GPU, that is not always the case in virtualized environments. The new canvas renderer, while much slower, now allows GenomeSpy to be used in these environments and may even be the preferred renderer for visualizations that show only a moderate amount of data on the screen.

For a long time, the official way to export figures was to take a screenshot, which was a bit embarrassing for a visualization toolkit. PNG export provided some relief, but now there is SVG export that produces publication-quality vector graphics with automatic rasterization of dense layers like heatmaps and large scatter plots. In addition, the visualization’s hierarchical structure is replicated in the resulting SVG groups, enabling easy editing in vector art apps.

What’s next

There are already several small but breaking changes waiting in the backlog, aimed at cleaning up a few design decisions. Now that v1.0 is out, it is clear that those changes will go into v2.0, which I expect to release in the relatively near future. v2.0 will come with a migration guide for the grammar and API changes. It will also include new long-awaited features enabled—or at least significantly simplified—by a WebGPU renderer, which is already feature-complete for the current grammar but needs further testing.

GenomeSpy 1.0 also brings something important on the Python side. I’ve been collaborating with Oskari Lehtonen, who recently created an Altair-like Python package for GenomeSpy, and it is reaching v1.0 at roughly the same time. A package like that has been on my TODO list for years, as JSON isn’t a typical bioinformatician’s mother tongue, while Python could actually be. It provides a much more convenient way to construct GenomeSpy visualizations in Python notebooks and analysis workflows.

The Python package removes one important barrier to using GenomeSpy, but it may not be a drop-in solution for many common workflows: the user still has to specify a visualization. Many tasks do not need that level of customization, and a predefined template—or a “track type”—would be perfectly adequate.

Figuring out which of these higher-level building blocks would actually be useful is one of the next steps. If you use GenomeSpy, or have looked at it and decided that it takes too much work to get started, I’d be interested to hear what you were trying to do. Perhaps some of those workflows should simply work out of the box.

— Kari Lavikka (linkedin)