DataFrame

DataFrame is a 2D, columnar table structure composed of multiple equal-length Series. It is the most commonly used data structure in Python query scripts — query() returns a DataFrame, and main() should preferably return a DataFrame.

Construction

DataFrame(data=None, schema=None, *, orient=None)

data supports multiple forms:

# Row-wise: list[dict]
DataFrame([{"a": 1, "b": "x"}, {"a": 2, "b": "y"}])

# Column-wise: dict[str, list]
DataFrame({"a": [1, 2], "b": ["x", "y"]})

# From Series
DataFrame([Series("a", [1, 2]), Series("b", ["x", "y"])])

• schema — optional, specifies column names and dtypes, e.g. {"a": pl.Int, "b": pl.String} or a list of column names.
• orient — "row" or "col", used when data is a nested list to specify orientation.

Properties

Property	Type	Description
columns	list[str]	Column names (in order)
dtypes	list[DataType]	Dtype of each column (in column order)
schema	dict[str, DataType]	Mapping of column name to dtype
shape	tuple[int, int]	(rows, columns)
height	int	Number of rows
width	int	Number of columns

Methods

is_empty

df.is_empty() -> bool

Whether the DataFrame has zero rows.

Selection

Method	Description
df.head(n=5)	First n rows (negative n drops the last |n| rows)
df.tail(n=5)	Last n rows (negative n drops the first |n| rows)
df.limit(n=5)	Alias for head
df.slice(offset, length=None)	Rows from offset (can be negative) for length rows (None = to end)
df.gather_every(n, offset=0)	Every n-th row, starting at offset
df.sample(n=None, *, fraction, with_replacement, shuffle, seed)	Random sample (n or fraction, not both)

Transformations

Method	Description
df.unique(subset=None, *, keep="first")	Drop duplicates by subset columns (default: all); keep: "first" / "last" / "any" / "none"
df.n_unique(subset=None)	Count of distinct rows (by subset, default all columns)
df.sort(by, *, descending=False, nulls_last=False)	Sort by one or more columns (stable); descending / nulls_last can be a single bool or a list
df.shift(n=1, *, fill_value)	Shift all columns by n positions; vacated positions become fill_value (default null)

Row-wise Aggregation (collapse to single row)

The following methods evaluate each column independently, collapsing the result to a single row. Columns that don't support the operation (e.g. strings) become null.

Method	Description
df.sum()	Sum of each column
df.mean()	Arithmetic mean of each column
df.min() / df.max()	Min / max of each column
df.median()	Median of each column
df.std() / df.var()	Sample std / variance of each column
df.count()	Non-null count of each column

select

df.select(*exprs) -> DataFrame

Evaluates expressions (or bare column names) into a new DataFrame (only the selected columns).

df.select(pl.col("name"), (pl.col("amount") * 2).alias("double"))

with_columns

df.with_columns(*exprs) -> DataFrame

Adds or overwrites columns while keeping all existing columns.

df.with_columns(pl.col("created_at").str.to_datetime())

filter

df.filter(predicate) -> DataFrame

Keeps only rows where predicate (a boolean Expr) evaluates to true.

df.filter(pl.col("amount") > 500)

group_by

df.group_by(*keys) -> GroupBy

Groups by one or more column names, returning a GroupBy view (used with .agg()).

df.group_by("category", "region").agg(pl.col("amount").sum().alias("total"))

group_by_dynamic (Time-window Grouping)

df.group_by_dynamic(
    index_column: str,
    *,
    every: str,
    period: str | None = None,
    offset: str | None = None,
    closed = "left",
    label = "left",
    group_by: str | list[str] | None = None,
) -> DynamicGroupBy

Groups into dynamic (rolling) time windows over index_column (must be Datetime or Date).

• every — window step (e.g. "2d", "1h")
• period — window width, defaults to every
• offset — offset for window boundaries
• closed — boundary inclusivity: "left" / "right" / "both" / "none"
• label — which boundary to use as the window index in output
• group_by — additional key columns for partitioning before windowing

# 7-day rolling window aggregation
df.group_by_dynamic("date", every="7d").agg(
    pl.col("amount").sum().alias("weekly_total")
)

upsample

df.upsample(time_column, *, every, group_by=None) -> DataFrame

Inserts missing rows to make time_column (Datetime or Date) a regular grid stepped by every. Missing rows have null in all other columns — use forward_fill / backward_fill afterwards to fill them.

df.upsample("ts", every="1h").with_columns(
    pl.col("value").forward_fill()
)

Join

Equi Join

df.join(
    other,
    on=None,
    how="inner",
    *,
    left_on=None,
    right_on=None,
    suffix="_right",
    nulls_equal=False,
    coalesce=None,
) -> DataFrame

Provide either on (same key on both sides) or left_on/right_on (different key names). how supports "inner" / "left" / "right" / "full" / "cross" / "semi" / "anti" ("outer" is an alias for "full").

# Inner join
df_a.join(df_b, on="id")
# Left join with different key names
df_a.join(df_b, left_on="a_id", right_on="b_id", how="left")
# Anti join: rows from left whose key is not in right
df_a.join(df_b, on="id", how="anti")

Non-equi Join

df.join_where(other, *predicates, suffix="_right") -> DataFrame

Inner join on arbitrary predicate expressions, AND-ed together. Internally builds a Cartesian product, then filters.

# Range matching
df_a.join_where(df_b, pl.col("a.value") >= pl.col("b.low"), pl.col("a.value") < pl.col("b.high"))

As-of Join

df.join_asof(
    other,
    *,
    on=None, left_on=None, right_on=None,
    by=None, by_left=None, by_right=None,
    strategy="backward",
    suffix="_right",
    tolerance=None,
    allow_exact_matches=True,
) -> DataFrame

Match on the nearest key value. strategy: "backward" / "forward" / "nearest". tolerance is a number or duration string (e.g. "1d").

# Nearest match by timestamp
df_a.join_asof(df_b, on="ts", strategy="backward", tolerance="1h")

Export

Method	Returns	Description
df.to_dicts()	list[dict]	Convert to list of row dicts
df.rows()	list[tuple]	Convert to list of row tuples
df.to_dict()	dict[str, list]	Convert to column-name → list-of-values
df.get_column(name)	Series	Get a single column by name
df[name]	Series	Get a single column by name (raises KeyError if missing)
len(df)	int	Number of rows (equivalent to height)

GroupBy

A grouped view of a DataFrame, produced by DataFrame.group_by(...).

agg

gb.agg(*exprs) -> DataFrame

Aggregates each group; every expression must reduce to a single value.

df.group_by("category").agg(
    pl.col("amount").sum().alias("total"),
    pl.col("amount").mean().alias("avg"),
    pl.col("id").n_unique().alias("orders"),
)

Shortcut Aggregations

The following methods are equivalent to calling .agg() on all non-key columns:

Method	Description
gb.sum()	Sum of each column
gb.mean()	Arithmetic mean of each column
gb.min() / gb.max()	Min / max of each column
gb.median()	Median of each column
gb.n_unique()	Distinct count of each column
gb.first() / gb.last()	First / last value of each column
gb.count()	Non-null count of each column
gb.quantile(q)	Quantile of each column

len

gb.len() -> DataFrame

Group sizes, returned as a two-column DataFrame of group keys plus a "len" count column.

map_groups

gb.map_groups(function) -> DataFrame

Applies function to each group (as a sub-DataFrame), concatenating the returned DataFrames. Each group's result must share the same column names.

# Top 2 rows by amount per category
df.group_by("category").map_groups(lambda g: g.sort("amount", descending=True).head(2))

DynamicGroupBy

A time-window grouped view produced by DataFrame.group_by_dynamic(...).

dg.agg(*exprs) -> DataFrame

Aggregates each time window. Output column order: group_by key columns (if any), the window index label column (named after index_column), then the aggregated columns.

df.group_by_dynamic("ts", every="1d").agg(
    pl.col("value").mean().alias("daily_avg"),
    pl.col("value").max().alias("daily_max"),
)