usage: parquet-to-blosc2 [-h] [--export | --roundtrip] [--parquet-batch-size PARQUET_BATCH_SIZE] [--fixed-str-maxlen FIXED_STR_MAXLEN] [--fixed-bytes-maxlen FIXED_BYTES_MAXLEN] [--max-rows MAX_ROWS] [--blosc2-batch-size BLOSC2_BATCH_SIZE] [--blosc2-items-per-block BLOSC2_ITEMS_PER_BLOCK] [--list-serializer {msgpack,arrow}] [--use-dict] [--float-trunc-prec BITS|COLUMN=BITS] [--timestamp-unit {s,ms,us,ns,auto}] [--chunks CHUNKS] [--blocks BLOCKS] [--codec {BLOSCLZ,LZ4,LZ4HC,ZLIB,ZSTD,NDLZ,ZFP_ACC,ZFP_PREC,ZFP_RATE,OPENHTJ2K,GROK,OPENZL,J2K,HTJ2K}] [--clevel CLEVEL] [--reduce-mem] [--mem-report] [--mem-every MEM_EVERY] [--batch-report-every BATCH_REPORT_EVERY] [--progress] [--profile] [--overwrite] [--decode-dictionaries] [--separate-nested-cols | --no-separate-nested-cols] [--no-summary-index] input_path [output_path] Import/export Parquet datasets via Blosc2 CTable (.b2z compact or .b2d sparse). positional arguments: input_path Input parquet file or Blosc2 store, depending on mode. output_path Output path. Defaults depend on the mode and input path. (default: None) options: -h, --help show this help message and exit --export Export input .b2z/.b2d to output parquet. (default: False) --roundtrip Run parquet -> .b2z/.b2d -> parquet and compare. (default: False) --parquet-batch-size PARQUET_BATCH_SIZE Rows per Parquet read batch. Defaults to the source Parquet average row- group size. (default: None) --fixed-str-maxlen FIXED_STR_MAXLEN Pre-scan string columns and import columns whose maximum character length is at most this value as fixed-width, indexable strings. Other string columns remain utf8. (default: None) --fixed-bytes-maxlen FIXED_BYTES_MAXLEN Pre-scan binary columns and import columns whose maximum byte length is at most this value as fixed-width, indexable bytes. Other binary columns remain vlbytes. (default: None) --max-rows MAX_ROWS Maximum number of CTable rows to import. In normal mode this equals the number of Parquet rows read. With separate nested columns enabled for an unnamed-root list> file, the unit is list elements (i.e. the number of rows in the resulting CTable), not outer Parquet rows. (default: None) --blosc2-batch-size BLOSC2_BATCH_SIZE Internal batch_rows for BatchArray/varlen columns in the imported CTable. Defaults to the blocks value from blosc2.compute_chunks_blocks() based on the estimated CTable row count. (default: None) --blosc2-items-per-block BLOSC2_ITEMS_PER_BLOCK Items per internal BatchArray block for imported Blosc2 varlen/list columns. Defaults to BatchArray's automatic heuristic. (default: None) --list-serializer {msgpack,arrow} Serializer for imported list columns (default: msgpack). MessagePack can compress simple lists better and avoids a PyArrow read-time dependency. Choose 'arrow' for direct Arrow ingestion/export, potentially at the cost of larger storage. (default: msgpack) --use-dict Enable C-Blosc2 dictionary compression. (default: False) --float-trunc-prec BITS|COLUMN=BITS Apply the Blosc2 TRUNC_PREC filter to imported float32/float64 columns. Pass an integer to affect all float columns, or COLUMN=integer for a single column. May be repeated; column-specific entries override the global value. (default: []) --timestamp-unit {s,ms,us,ns,auto} Import timestamp columns using this unit. Explicit units use Arrow's safe cast and fail if conversion would lose precision. 'auto' pre-scans timestamp columns and chooses the coarsest lossless unit per column. (default: None) --chunks CHUNKS Chunk size (in rows) for all scalar columns in the imported CTable. Overrides the automatic chunk size chosen by blosc2.compute_chunks_blocks(). Only affects fixed-width scalar columns; list, varlen, and dictionary columns use their own internal chunking. (default: None) --blocks BLOCKS Block size (in rows) for all scalar columns in the imported CTable. Overrides the automatic block size chosen by blosc2.compute_chunks_blocks(). Must be <= chunks; if omitted when --chunks is given, blosc2 picks a suitable block size. (default: None) --codec {BLOSCLZ,LZ4,LZ4HC,ZLIB,ZSTD,NDLZ,ZFP_ACC,ZFP_PREC,ZFP_RATE,OPENHTJ2K,GROK,OPENZL,J2K,HTJ2K} --clevel CLEVEL --reduce-mem Shrink an auto-chosen Parquet batch size so a single Arrow read batch fits a small memory budget, lowering peak RSS at the cost of import speed. Only affects auto batch sizing for unnamed-root list> flattening; an explicit --batch-size is always left untouched. (default: False) --mem-report Print process/Arrow memory diagnostics at import phases and during batch processing. (default: False) --mem-every MEM_EVERY With --mem-report, print batch memory diagnostics every N batches. (default: 1) --batch-report-every BATCH_REPORT_EVERY With --progress, print progress every N batches; the final batch is always reported. (default: 1) --progress Print import progress lines. By default, only the import summary is shown. (default: False) --profile Run the selected operation under cProfile and print cumulative timing stats. (default: False) --overwrite --decode-dictionaries Decode Arrow dictionary-encoded columns to plain utf8 instead of preserving the dictionary encoding. By default, supported dictionary columns (string values with integer indices) are imported as Blosc2 dictionary columns. (default: False) --separate-nested-cols, --no-separate-nested-cols Import nested columns as separate CTable columns where possible. Top-level struct fields are flattened recursively into dotted leaf columns (e.g. trip.begin.lon). For a single unnamed top-level list> field (the Awkward Array / Chicago-taxi layout), flatten the outer list so that each element becomes a CTable row. Enabled by default; use --no-separate- nested-cols when closer Parquet schema fidelity is desired. (default: True) --no-summary-index Disable automatic SUMMARY index creation on close. By default, SUMMARY indexes are built for all eligible scalar columns, which costs <0.1% of column size and accelerates WHERE queries. (default: True)