Data sources¶
Pandas & Polars DataFrames¶
import pandas as pd
import polars as pl
from rustpy_xlsxwriter import FastExcel
# Pandas — Arrow zero-copy, dtype-aware
df_pd = pd.DataFrame({"Name": ["Alice", "Bob"], "Score": [88.5, 92.3]})
FastExcel("pandas.xlsx").sheet("Data", df_pd).save()
# Polars — native support, no .to_pandas() needed
df_pl = pl.DataFrame({"Name": ["Alice", "Bob"], "Score": [88.5, 92.3]})
FastExcel("polars.xlsx").sheet("Data", df_pl).save()
Selecting columns¶
columns picks and orders the columns to write, for xlsx as for CSV, without
slicing the data in Python first:
write_worksheet(df, "out.xlsx", columns=["sku", "name"])
FastExcel("out.xlsx").sheet("Stock", rows, columns=["sku", "qty"]).save()
write_worksheets([("A", a), ("B", b)], "out.xlsx", columns={"general": ["sku"]})
It works on every input type and stays zero-copy on the Arrow path. A name
that is not in the data, or one listed twice, raises ValueError. A key a
record carries outside the list is dropped on purpose. Options keyed by
column name refer to the selected columns, and formula_columns follow them,
so a {row} formula's column letters count from the selection. A column
whose type cannot be written is only refused if it is selected.
Generator Streaming¶
def rows():
for i in range(1_000_000):
yield {"id": i, "value": f"row_{i}"}
FastExcel("streamed.xlsx").sheet("Data", rows()).save()
Note:
dedupe_strings=Truebuffers the sheet, so it defeats the point of generator streaming. Leave it off for very large streamed exports.
In-Memory Buffer (Web Frameworks)¶
import io
from rustpy_xlsxwriter import FastExcel
buf = io.BytesIO()
FastExcel(buf).sheet("Sheet1", records).save()
xlsx_bytes = buf.getvalue() # send as HTTP response
String Deduplication¶
By default every sheet is written in constant-memory mode: strings go inline
into the sheet XML and nothing is buffered. Passing dedupe_strings=True takes
that sheet out of constant-memory mode and stores each distinct string once in
the workbook's shared-string table instead.
(
FastExcel("report.xlsx")
.sheet("Events", events, dedupe_strings=True) # lots of repeated text
.sheet("Raw", raw_rows) # default: streamed
.save()
)
Measured on 50k rows — the uncompressed XML shrinks a lot, but .xlsx is a zip
and deflate already collapses repetition, so the on-disk win is modest and
can even be negative:
| Data | Uncompressed | On disk | Write time |
|---|---|---|---|
| Short repeats (4 distinct values) | −11% | +2% | 1.7x |
| Long repeats (20 distinct, 120 chars) | −56% | −5% | 1.1x |
| All-unique strings | +11% | −1% | 1.5x |
| 20 repeated text columns | −39% | −9% | 1.1x |
Worth enabling for sheets with many repeated long strings, or when the consumer parses the uncompressed XML. Not worth it for short categorical values. Off by default because it buffers the whole sheet in memory — measure on your own data before turning it on for a large export.
For write_worksheets, pass dedupe_strings as a dict keyed by sheet name
(with a "general" fallback key).
Very large workbooks¶
A plain zip entry stops at 4 GB uncompressed, which is around 150 million cells — a million rows across 185 columns reaches it. Every workbook is therefore written as ZIP64, so that limit does not apply. There is nothing to switch on: it cannot be decided once the limit is hit, because a sheet's final size is unknown until it has been written and the zip is streamed, so it is always on. The cost is a flat 432 bytes per file.
Excel's own limits still apply, and those are reported: 1,048,576 rows and 16,384 columns per sheet, and 32,767 characters per cell.
Compression level¶
An .xlsx is a zip, and compressing the sheet XML is most of what saving
costs — on a multi-sheet workbook, 59% of the whole write. compression_level
is the knob on it, 1 to 9, default 6:
write_worksheets(sheets, "report.xlsx", compression_level=3)
Measured on four sheets of 200k rows × 5 columns:
| Level | Write time | On disk | Speedup |
|---|---|---|---|
| 1 | 1.34s | 32.2 MB | 1.75x |
| 3 | 1.63s | 20.7 MB | 1.44x |
| 6 (default) | 2.35s | 18.9 MB | 1.00x |
| 9 | 5.99s | 18.8 MB | 0.39x |
3 is the one to reach for: a third off the write for 9% more file. 1 pays
70% more disk for half the time, which is worth it for a temporary file or
something about to be re-compressed in transit.
9 is never worth it. It costs two and a half times the default's write
time to save 0.5% of the file, because the XML is already highly repetitive
and deflate has little left to find. It is accepted so the full range is
available, not because there is a reason to use it.
A level outside 1–9 raises ValueError rather than falling back to the
default, which is what the underlying crate does with only a line on stderr.