Skip to main content

Python Query Scripts

Python query scripts are the third query script type alongside SQL and DQL. They let you write data processing logic in real Python with standard libraries: query data sources like SQL, transform/aggregate/clean data with a Polars-style DataFrame API, and pull external data via fetch().

Like other modes in the query editor, Python scripts follow the same execution workflow — select a data source, write a script, run, view results, and visualize.

note

DQL is based on Starlark (a Python-like language, but not actual Python). Python query scripts run real Python (interpreted by RustPython compiled to WebAssembly, executed in a sandbox), so you can import standard libraries and use full Python syntax. The data processing API uses a Polars-style design (pl.col, select/with_columns/filter/group_by().agg() with lazy expressions).

Quick Start​

Python scripts must define a main() function as the entry point; its return value is the query result:

def main():
    return [{"city": "Beijing", "value": 1}, {"city": "Shanghai", "value": 2}]

main() can return:

  • DataFrame — directly converted to result set (recommended, zero JSON round-trip)
  • Series — converted to a single-column result set
  • list[dict] or other JSON-serializable values — go through JSON conversion

No import needed: DataFrame, Series, pl, as well as query, fetch, args, print are all directly injected into the script scope.

Querying Data Sources​

Use the built-in query() to execute SQL, returning a DataFrame:

def main():
    df = query("SELECT id, name, amount FROM orders WHERE amount > ?", 100)
    return df.filter(pl.col("amount") > 500)

query() uses the same query engine pathway as regular SQL queries, supporting bound/mounted data sources and cross-source queries.

warning

SQL timestamp / date columns are returned as strings. Convert them explicitly:

df = df.with_columns(pl.col("created_at").str.to_datetime())

Using Parameters​

Caller-supplied parameters are accessed via the global args (always a dict):

def main():
    threshold = args["threshold"]
    df = query("SELECT * FROM sales")
    return df.filter(pl.col("revenue") > threshold)

Making HTTP Requests​

The built-in fetch() provides JS fetch-style HTTP access:

def main():
    res = fetch("https://api.example.com/data", headers={"Authorization": "Bearer ..."})
    if not res.ok:
        return [{"error": res.status}]
    return res.json()
  • body as dict / list is auto-serialized to JSON with default content-type: application/json
  • Transport errors (DNS / connection failure) raise exceptions; HTTP 4xx / 5xx do NOT raise — check res.ok / res.status
  • res.text() for raw text, res.json() for JSON parsing, res.headers.get(name) for response headers

Data Processing (Polars-style)​

DataFrame / Series / lazy Expr provide selection, filtering, derived columns, group-by aggregation, and more. Also supported: rolling windows, exponentially weighted moving, time-window grouping, joins, conditional expressions, and cumulative operations:

def main():
    df = query("SELECT category, region, amount FROM sales")
    return (
        df.filter(pl.col("amount").is_not_null())
          .group_by("category", "region")
          .agg(pl.col("amount").sum().alias("total"))
    )

See the Python Language Reference for the full API.

Logging​

The built-in print() is redirected to the script log (it does not pollute the result set), useful for debugging:

def main():
    df = query("SELECT * FROM orders")
    print("Row count:", df.height)
    return df

Sandbox & Limitations​

Python scripts execute in a sandboxed WebAssembly environment with the following constraints:

ConstraintDescription
Entry functionMust define a callable main(), otherwise the script errors
Execution timeoutMax 60 seconds per execution
Memory limitApprox. 256 MiB linear memory
No filesystemScripts cannot read or write local files
Result truncationResults exceeding a certain row count are truncated

Regarding standard libraries: frozen pure-Python libraries (such as json, re, argparse) and native modules like math, datetime, struct, hashlib can all be imported normally.