go-dataframe documentation

go-dataframe is labelled, columnar tables in pure Go. Series, DataFrame and Index, with selection, missing data, group-by, joins, reshaping, time series, a fast CSV reader, Arrow IPC and Parquet, notebook display and plots. Numeric columns are go-ndarray arrays, handed out without copying.

pandas is the behavioural reference: the operations give pandas’ answers (values, dtypes, index, row order), checked against pandas 3.0 on 173 recorded cases; the differences that remain are deliberate and listed in Design.

package main

import (
	"github.com/go-dataframe/dataframe"
	"github.com/go-dataframe/dataframe/dfplot"
	"github.com/go-plot/plot"
)

func init() { register("bars", bars) }

// df.groupby("city")[["rain"]].mean().plot.bar()
func bars() *plot.Figure {
	g := dataframe.Must(rainfall().GroupBy("city"))
	means := dataframe.Must(g.Mean())
	fig, err := dfplot.BarOpts(means, []string{"rain"}, []dfplot.Option{dfplot.WithTitle("Mean yearly rainfall, 2022–2024 (mm)")})
	if err != nil {
		panic(err)
	}
	return fig
}

These pages describe go-dataframe/dataframe v0.1.0. Every table printed and every figure in them was produced by that version, from the code shown beside it.

Install

go get github.com/go-dataframe/dataframe@v0.1.0

The module needs Go 1.27.2 or later. It is pure Go (CGO_ENABLED=0).

import "github.com/go-dataframe/dataframe"

df := dataframe.Must(dataframe.ReadCSVFile("weather.csv", dataframe.CSVParseDates("date")))
fmt.Println(df.Head(5))

The pages

Speed, honestly

Medians of interleaved runs on 10⁷ rows (Apple M4 Max, a shared host under load; the performance page has the method, a second run under heavier load, 10⁶ rows, and Arrow/Parquet). Milliseconds:

go-dataframe is faster than pandas on all of these but the string-key group-by, which is at parity (CSV by about 10x, sorting by 2x); polars remains faster on most of them. Memory is at par with pandas for numeric columns but worse for strings (a Go []string costs 16 bytes a value; pandas 3 and polars use Arrow-style string buffers): the benchmark table takes 400 MB here, 349 MB in pandas, 269 MB in polars.

Packages

Verification

• testdata/gen_fixtures.py records pandas’ answers (values, dtypes, index, NA handling, sort stability, group-by order, merge row order, repr text) for 173 cases; TestPandasFixtures replays them.
• A corpus of 47 awkward CSV files (69 option sets) is compared with pandas' parse, and Arrow and Parquet files written by pandas/pyarrow are read back and compared.
• 100% statement coverage of the whole module under the race detector, on Linux, macOS and Windows; tests also run natively on amd64/arm64 and under qemu on riscv64, loong64, ppc64le and s390x (big-endian).

Built on

go-ndarray (numeric columns), go-plot (dfplot) and apache/arrow-go (Arrow and Parquet). A go-notebook cell shows a frame through Display.