usage: sacrebleu [-h] [--citation] [--list] [--test-set TEST_SET] [--language-pair LANGPAIR] [--origlang ORIGLANG] [--subset SUBSET] [--download DOWNLOAD] [--echo ECHO [ECHO ...]] [--input [INPUT ...]] [--num-refs NUM_REFS] [--encoding ENCODING] [--metrics {bleu,chrf,ter} [{bleu,chrf,ter} ...]] [--sentence-level] [--smooth-method {none,floor,add-k,exp}] [--smooth-value BLEU_SMOOTH_VALUE] [--tokenize {none,zh,13a,intl,char,ja-mecab,ko-mecab,spm,flores101,flores200,spBLEU-1K}] [--lowercase] [--force] [--chrf-char-order CHRF_CHAR_ORDER] [--chrf-word-order CHRF_WORD_ORDER] [--chrf-beta CHRF_BETA] [--chrf-whitespace] [--chrf-lowercase] [--chrf-eps-smoothing] [--ter-case-sensitive] [--ter-asian-support] [--ter-no-punct] [--ter-normalized] [--confidence] [--confidence-n CONFIDENCE_N] [--paired-ar | --paired-bs] [--paired-ar-n PAIRED_AR_N] [--paired-bs-n PAIRED_BS_N] [--paired-jobs PAIRED_JOBS] [--quiet] [--short] [--score-only] [--width WIDTH] [--detail] [--no-color] [--format {json,text,latex}] [--version] [refs ...] sacreBLEU: Hassle-free computation of shareable BLEU scores. Quick usage: score your detokenized output against WMT'14 EN-DE: cat output.detok.de | sacrebleu -t wmt14 -l en-de positional arguments: refs Optional list of references. If given, it should preceed the -i/--input argument. options: -h, --help show this help message and exit --citation, --cite Dump the bibtex citation and quit. --list Print a list of all available test sets. --test-set TEST_SET, -t TEST_SET The test set to use (see also --list) or a comma-separated list of test sets to be concatenated. --language-pair LANGPAIR, -l LANGPAIR Source-target language pair (2-char ISO639-1 codes). --origlang ORIGLANG, -ol ORIGLANG Use a subset of sentences with a given original language (2-char ISO639-1 codes), "non-" prefix means negation. --subset SUBSET Use a subset of sentences whose document annotation matches a given regex (see SUBSETS in the source code). --download DOWNLOAD Download a test set and quit. --echo ECHO [ECHO ...] Output the source (src), reference (ref), or other available field (docid, ref:A, ref:1 for example) to STDOUT and quit. You can get available fields with options `--list` and `-t`For example: `sacrebleu -t wmt21 --list`. If multiple fields are given, they are outputted with tsv format in the order they are given.You can also use `--echo all` to output all available fields. --input [INPUT ...], -i [INPUT ...] Read input from file(s) instead of STDIN. --num-refs NUM_REFS, -nr NUM_REFS Split the reference stream on tabs, and expect this many references. (Default: 1) --encoding ENCODING, -e ENCODING Open text files with specified encoding (Default: utf-8) --metrics {bleu,chrf,ter} [{bleu,chrf,ter} ...], -m {bleu,chrf,ter} [{bleu,chrf,ter} ...] Space-delimited list of metrics to compute (Default: bleu) --sentence-level, -sl Compute metric for each sentence. --version, -V show program's version number and exit BLEU related arguments: --smooth-method {none,floor,add-k,exp}, -s {none,floor,add-k,exp} Smoothing method: exponential decay, floor (increment zero counts), add-k (increment num/denom by k for n>1), or none. (Default: exp) --smooth-value BLEU_SMOOTH_VALUE, -sv BLEU_SMOOTH_VALUE The smoothing value. Only valid for floor and add-k. (Defaults: floor: 0.1, add-k: 1) --tokenize {none,zh,13a,intl,char,ja-mecab,ko-mecab,spm,flores101,flores200,spBLEU-1K}, -tok {none,zh,13a,intl,char,ja-mecab,ko-mecab,spm,flores101,flores200,spBLEU-1K} Tokenization method to use for BLEU. If not provided, defaults to `zh` for Chinese, `ja-mecab` for Japanese, `ko-mecab` for Korean and `13a` (mteval) otherwise. --lowercase, -lc If True, enables case-insensitivity. (Default: False) --force Insist that your tokenized input is actually detokenized. chrF related arguments: --chrf-char-order CHRF_CHAR_ORDER, -cc CHRF_CHAR_ORDER Character n-gram order. (Default: 6) --chrf-word-order CHRF_WORD_ORDER, -cw CHRF_WORD_ORDER Word n-gram order (Default: 0). If equals to 2, the metric is referred to as chrF++. --chrf-beta CHRF_BETA Determine the importance of recall w.r.t precision. (Default: 2) --chrf-whitespace Include whitespaces when extracting character n-grams. (Default: False) --chrf-lowercase Enable case-insensitivity. (Default: False) --chrf-eps-smoothing Enables epsilon smoothing similar to chrF++.py, NLTK and Moses; instead of effective order smoothing. (Default: False) TER related arguments (The defaults replicate TERCOM's behavior): --ter-case-sensitive Enables case sensitivity. (Default: False) --ter-asian-support Enables special treatment of Asian characters. (Default: False) --ter-no-punct Removes punctuation. (Default: False) --ter-normalized Applies basic normalization and tokenization. (Default: False) Confidence interval (CI) estimation for single-system evaluation: --confidence, -ci Report confidence interval using bootstrap resampling. --confidence-n CONFIDENCE_N, -cin CONFIDENCE_N Set the number of bootstrap resamples for CI estimation (Default: 1000). Paired significance testing for multi-system evaluation: --paired-ar, -par Perform paired test using approximate randomization (AR). This option is mutually exclusive with --paired-bs (Default: False). --paired-bs, -pbs Perform paired test using bootstrap resampling. This option is mutually exclusive with --paired-ar (Default: False). --paired-ar-n PAIRED_AR_N, -parn PAIRED_AR_N Number of trials for approximate randomization test (Default: 10000). --paired-bs-n PAIRED_BS_N, -pbsn PAIRED_BS_N Number of bootstrap resamples for paired bootstrap resampling test (Default: 1000). --paired-jobs PAIRED_JOBS, -j PAIRED_JOBS If 0, launches as many workers as the number of systems. If > 0, sets the number of workers manually. This feature is currently not supported on Windows. Reporting related arguments: --quiet, -q Suppress verbose messages. --short, -sh Produce a shorter (less human readable) signature. --score-only, -b Print only the computed score. --width WIDTH, -w WIDTH Floating point width (Default: 1). --detail, -d Print detailed information (split test sets based on origlang). --no-color, -nc Disable the occasional use of terminal colors. --format {json,text,latex}, -f {json,text,latex} Set the output format. `latex` is only valid for multi-system mode whereas `json` and `text` apply to single-system mode only. This flag is overridden if the SACREBLEU_FORMAT environment variable is set to one of the valid choices (Default: json).