/usr/bin/parquet-to-blosc2

usage: parquet-to-blosc2 [-h] [--export | --roundtrip] [--parquet-batch-size PARQUET_BATCH_SIZE]
                         [--fixed-str-maxlen FIXED_STR_MAXLEN]
                         [--fixed-bytes-maxlen FIXED_BYTES_MAXLEN] [--max-rows MAX_ROWS]
                         [--blosc2-batch-size BLOSC2_BATCH_SIZE]
                         [--blosc2-items-per-block BLOSC2_ITEMS_PER_BLOCK]
                         [--list-serializer {msgpack,arrow}] [--use-dict]
                         [--float-trunc-prec BITS|COLUMN=BITS]
                         [--timestamp-unit {s,ms,us,ns,auto}] [--chunks CHUNKS] [--blocks BLOCKS]
                         [--codec {BLOSCLZ,LZ4,LZ4HC,ZLIB,ZSTD,NDLZ,ZFP_ACC,ZFP_PREC,ZFP_RATE,OPENHTJ2K,GROK,OPENZL,J2K,HTJ2K}]
                         [--clevel CLEVEL] [--reduce-mem] [--mem-report] [--mem-every MEM_EVERY]
                         [--batch-report-every BATCH_REPORT_EVERY] [--progress] [--profile]
                         [--overwrite] [--decode-dictionaries]
                         [--separate-nested-cols | --no-separate-nested-cols] [--no-summary-index]
                         input_path [output_path]

Import/export Parquet datasets via Blosc2 CTable (.b2z compact or .b2d sparse).

positional arguments:
  input_path            Input parquet file or Blosc2 store, depending on mode.
  output_path           Output path. Defaults depend on the mode and input path. (default: None)

options:
  -h, --help            show this help message and exit
  --export              Export input .b2z/.b2d to output parquet. (default: False)
  --roundtrip           Run parquet -> .b2z/.b2d -> parquet and compare. (default: False)
  --parquet-batch-size PARQUET_BATCH_SIZE
                        Rows per Parquet read batch. Defaults to the source Parquet average row-
                        group size. (default: None)
  --fixed-str-maxlen FIXED_STR_MAXLEN
                        Pre-scan string columns and import columns whose maximum character length
                        is at most this value as fixed-width, indexable strings. Other string
                        columns remain utf8. (default: None)
  --fixed-bytes-maxlen FIXED_BYTES_MAXLEN
                        Pre-scan binary columns and import columns whose maximum byte length is at
                        most this value as fixed-width, indexable bytes. Other binary columns
                        remain vlbytes. (default: None)
  --max-rows MAX_ROWS   Maximum number of CTable rows to import. In normal mode this equals the
                        number of Parquet rows read. With separate nested columns enabled for an
                        unnamed-root list<struct<...>> file, the unit is list elements (i.e. the
                        number of rows in the resulting CTable), not outer Parquet rows. (default:
                        None)
  --blosc2-batch-size BLOSC2_BATCH_SIZE
                        Internal batch_rows for BatchArray/varlen columns in the imported CTable.
                        Defaults to the blocks value from blosc2.compute_chunks_blocks() based on
                        the estimated CTable row count. (default: None)
  --blosc2-items-per-block BLOSC2_ITEMS_PER_BLOCK
                        Items per internal BatchArray block for imported Blosc2 varlen/list
                        columns. Defaults to BatchArray's automatic heuristic. (default: None)
  --list-serializer {msgpack,arrow}
                        Serializer for imported list columns (default: msgpack). MessagePack can
                        compress simple lists better and avoids a PyArrow read-time dependency.
                        Choose 'arrow' for direct Arrow ingestion/export, potentially at the cost
                        of larger storage. (default: msgpack)
  --use-dict            Enable C-Blosc2 dictionary compression. (default: False)
  --float-trunc-prec BITS|COLUMN=BITS
                        Apply the Blosc2 TRUNC_PREC filter to imported float32/float64 columns.
                        Pass an integer to affect all float columns, or COLUMN=integer for a
                        single column. May be repeated; column-specific entries override the
                        global value. (default: [])
  --timestamp-unit {s,ms,us,ns,auto}
                        Import timestamp columns using this unit. Explicit units use Arrow's safe
                        cast and fail if conversion would lose precision. 'auto' pre-scans
                        timestamp columns and chooses the coarsest lossless unit per column.
                        (default: None)
  --chunks CHUNKS       Chunk size (in rows) for all scalar columns in the imported CTable.
                        Overrides the automatic chunk size chosen by
                        blosc2.compute_chunks_blocks(). Only affects fixed-width scalar columns;
                        list, varlen, and dictionary columns use their own internal chunking.
                        (default: None)
  --blocks BLOCKS       Block size (in rows) for all scalar columns in the imported CTable.
                        Overrides the automatic block size chosen by
                        blosc2.compute_chunks_blocks(). Must be <= chunks; if omitted when
                        --chunks is given, blosc2 picks a suitable block size. (default: None)
  --codec {BLOSCLZ,LZ4,LZ4HC,ZLIB,ZSTD,NDLZ,ZFP_ACC,ZFP_PREC,ZFP_RATE,OPENHTJ2K,GROK,OPENZL,J2K,HTJ2K}
  --clevel CLEVEL
  --reduce-mem          Shrink an auto-chosen Parquet batch size so a single Arrow read batch fits
                        a small memory budget, lowering peak RSS at the cost of import speed. Only
                        affects auto batch sizing for unnamed-root list<struct<...>> flattening;
                        an explicit --batch-size is always left untouched. (default: False)
  --mem-report          Print process/Arrow memory diagnostics at import phases and during batch
                        processing. (default: False)
  --mem-every MEM_EVERY
                        With --mem-report, print batch memory diagnostics every N batches.
                        (default: 1)
  --batch-report-every BATCH_REPORT_EVERY
                        With --progress, print progress every N batches; the final batch is always
                        reported. (default: 1)
  --progress            Print import progress lines. By default, only the import summary is shown.
                        (default: False)
  --profile             Run the selected operation under cProfile and print cumulative timing
                        stats. (default: False)
  --overwrite
  --decode-dictionaries
                        Decode Arrow dictionary-encoded columns to plain utf8 instead of
                        preserving the dictionary encoding. By default, supported dictionary
                        columns (string values with integer indices) are imported as Blosc2
                        dictionary columns. (default: False)
  --separate-nested-cols, --no-separate-nested-cols
                        Import nested columns as separate CTable columns where possible. Top-level
                        struct fields are flattened recursively into dotted leaf columns (e.g.
                        trip.begin.lon). For a single unnamed top-level list<struct<…>> field (the
                        Awkward Array / Chicago-taxi layout), flatten the outer list so that each
                        element becomes a CTable row. Enabled by default; use --no-separate-
                        nested-cols when closer Parquet schema fidelity is desired. (default:
                        True)
  --no-summary-index    Disable automatic SUMMARY index creation on close. By default, SUMMARY
                        indexes are built for all eligible scalar columns, which costs <0.1% of
                        column size and accelerates WHERE queries. (default: True)