USAGE re2py [ OPTIONS ] [ WARNINGS ] INPUT Input can be either a file or - for stdin. OPTIONS -? --help -h Show help message. --api Specify the API used by the generated code to interface with used- defined code. Option simple should be used in simple cases when there's no need for buffer refilling and storing lexer state. Option record should be used when lexer state needs to be stored in a record (struct, class, etc.). Option generic should be used in complex cases when the other two APIs are not flexible enough. --bit-vectors -b Optimize conditional jumps using bit masks. This option implies --nested-ifs. --captures, --leftmost-captures Enable submatch extraction with leftmost greedy capturing groups. The result is collected into an array yybmatch of capacity 2 * YYMAXNMATCH, and yynmatch is set to the number of groups for the matching rule. --captvars, --leftmost-captvars Enable submatch extraction with leftmost greedy capturing groups. The result is collected into variables yytl, yytr for k-th capturing group. --case-insensitive Treat single-quoted and double-quoted strings as case-insensitive. --case-inverted Invert the meaning of single-quoted and double-quoted strings: treat single-quoted strings as case-sensitive and double-quoted strings as case-insensitive. --case-ranges Collapse consecutive cases in a switch statements into a range of the form low ... high. This syntax is a C/C++ language extension that is supported by compilers like GCC, Clang and Tcc. The main advantage over using single cases is smaller generated code and faster generation time, although for some compilers like Tcc it also results in smaller binary size. --computed-gotos -g Optimize conditional jumps using non-standard "computed goto" extension (which must be supported by the compiler). re2py generates jump tables only in complex cases with a lot of conditional branches. Complexity threshold can be configured with cgoto:threshold configuration. Relative offsets can be enabled with cgoto:relative configuration. This option implies --bit-vectors. --computed-gotos-relative Similar to --computed-gotos but generate relative offsets for jump tables instead (which must be supported by the compiler). This option implies --computed-gotos. --conditions --start-conditions -c Enable support of Flex-like "conditions": multiple interrelated lexers within one block. This is an alternative to manually specifying different re2py blocks connected with goto or function calls. --depfile FILE Write dependency information to FILE in the form of a Makefile rule : [include-file ...]. This allows one to track build dependencies in the presence of include blocks/directives, so that updating include files triggers regeneration of the output file. This option depends on the --output option. --ebcdic --ecb -e Generate a lexer that reads input in EBCDIC encoding. re2py assumes that the character range is 0 -- 0xFF and character size is 1 byte. --empty-class Define the way re2py treats empty character classes. With match-empty (the default) empty class matches empty input (which is illogical, but backwards-compatible). With match-none empty class always fails to match. With error empty class raises a compilation error. --encoding-policy Define the way re2py treats Unicode surrogates. With fail re2py aborts with an error when a surrogate is encountered. With substitute re2py silently replaces surrogates with the error code point 0xFFFD. With ignore (the default) re2py treats surrogates as normal code points. The Unicode standard says that standalone surrogates are invalid, but real- world libraries and programs behave in different ways. --flex-syntax -F Partial support for Flex syntax: in this mode named definitions don't need the equal sign and the terminating semicolon, and when used they must be surrounded with curly braces. Names without curly braces are treated as double-quoted strings. --goto-label Use "goto/label" code model: encode DFA in form of labeled code blocks connected with goto transitions across blocks. This is only supported for languages that have a goto statement. --header --type-header -t HEADER Generate a HEADER file. The contents of the file can be specified using special blocks header:on and header:off. If conditions are used, the generated header will have a condition enum automatically appended to it (unless there is an explicit conditions block). -I PATH Add PATH to the list of locations which are used when searching for include files. This option is useful in combination with include block or directive. re2py looks for FILE in the directory of the parent file and in the include locations specified with -I option. --input Deprecated alias for --api. Option default corresponds to simple (it is indeed the default for most backends, but not for all). Option custom corresponds to generic. --input-encoding Specify the way re2py parses regular expressions. With ascii (the default) re2py handles input as ASCII-encoded: any sequence of code units is a sequence of standalone 1-byte characters. With utf8 re2py handles input as UTF8-encoded and recognizes multibyte characters. --invert-captures Invert the meaning of capturing and non-capturing groups. By default (...) is capturing and (! ...) is non-capturing. With this option (! ...) is capturing and (...) is non-capturing. --lang Specify the target language. Supported languages are C, D, Go, Haskell, Java, JS, OCaml, Python, Rust, Swift, V, Zig (more languages can be added via user-defined syntax files, see the --syntax option). Option none disables default suntax configs, so that the target language is undefined. --location-format Specify location format in messages. With gnu locations are printed as 'filename:line:column: ...'. With msvc locations are printed as 'filename(line,column) ...'. The default is gnu. --loop-switch Use "loop/switch" code model: encode DFA in form of a loop over a switch statement, where individual states are switch cases. State is stored in a variable yystate. Transitions between states update yystate to the case label of the destination state and continue execution to the head of the loop. --computed-continue This option requires the --loop-switch code model. It uses computed continue statements to select the next switch case directly, enabling jump-table optimization similar to --computed-gotos in the goto/label code model. --nested-ifs -s Use nested if statements instead of switch statements in conditional jumps. This usually results in more efficient code with non-optimizing compilers. --no-debug-info -i Do not output line directives. This may be useful when the generated code is stored in a version control system (to avoid huge autogenerated diffs on small changes). --no-generation-date Suppress date output in the generated file. --no-version Suppress version output in the generated file. --no-unsafe Do not generate unsafe wrapper over YYPEEK (this option is specific to Rust). For performance reasons YYPEEK should avoid bounds-checking, as the lexer already performs end-of-input checks in a more efficient way. The user may choose to provide a safe YYPEEK definition, or a definition that is unsafe only in release builds, in which case the --no-unsafe option helps to avoid warnings about redundant unsafe blocks. --output -o OUTPUT Specify the OUTPUT file. --posix-captures, -P Enable submatch extraction with POSIX-style capturing groups. The result is collected into an array yybmatch of capacity 2 * YYMAXNMATCH, and yynmatch is set to the number of groups for the matching rule. --posix-captvars Enable submatch extraction with POSIX-style capturing groups. The result is collected into variables yytl, yytr for k-th capturing group. --recursive-functions Use code model based on co-recursive functions, where each DFA state is a separate function that may call other state-functions or itself. --reusable -r Deprecated since version 2.2 (reusable blocks are allowed by default now). --skeleton -S Ignore user-defined interface code and generate a self-contained "skeleton" program. Additionally, generate input files with strings derived from the regular grammar and compressed match results that are used to verify "skeleton" behavior on all inputs. This option is useful for finding bugs in optimizations and code generation. This option is supported only for C. --storable-state -f Generate a lexer which can store its inner state. This is useful in push-model lexers which are stopped by an outer program when there is not enough input, and then resumed when more input becomes available. In this mode users should additionally define YYGETSTATE and YYSETSTATE primitives, and variables yych, yyaccept and state should be part of the stored lexer state. --syntax FILE Load configurations from the specified FILE and apply them on top of the default syntax file. Note that FILE can define only a few configurations (if it's used to amend the default syntax file), or it can define a whole new language backend (in the latter case it is recommended to use --lang none option). --tags -T Enable submatch extraction with tags. --ucs2 --wide-chars -w Generate a lexer that reads UCS2-encoded input. re2py assumes that the character range is 0 -- 0xFFFF and character size is 2 bytes. This option implies --nested-ifs. --utf8 --utf-8 -8 Generate a lexer that reads input in UTF-8 encoding. re2py assumes that the character range is 0 -- 0x10FFFF and character size is 1 byte. --utf16 --utf-16 -x Generate a lexer that reads UTF16-encoded input. re2py assumes that the character range is 0 -- 0x10FFFF and character size is 2 bytes. This option implies --nested-ifs. --utf32 --unicode -u Generate a lexer that reads UTF32-encoded input. re2py assumes that the character range is 0 -- 0x10FFFF and character size is 4 bytes. This option implies --nested-ifs. --verbose Output a short message in case of success. --vernum -V Show version information in MMmmpp format (major, minor, patch). --version -v Show version information. --single-pass -1 Deprecated. Does nothing (single pass is the default now). --debug-output -d Emit YYDEBUG invocations in the generated code. This is useful to trace lexer execution. --dump-adfa Debug option: output DFA after tunneling (in .dot format). --dump-cfg Debug option: output control flow graph of tag variables (in .dot format). --dump-closure-stats Debug option: output statistics on the number of states in closure. --dump-dfa-det Debug option: output DFA immediately after determinization (in .dot format). --dump-dfa-min Debug option: output DFA after minimization (in .dot format). --dump-dfa-tagopt Debug option: output DFA after tag optimizations (in .dot format). --dump-dfa-tree Debug option: output DFA under construction with states represented as tag history trees (in .dot format). --dump-dfa-raw Debug option: output DFA under construction with expanded state-sets (in .dot format). --dump-interf Debug option: output interference table produced by liveness analysis of tag variables. --dump-nfa Debug option: output NFA (in .dot format). --emit-dot -D Instead of normal output generate lexer graph in .dot format. The output can be converted to an image with the help of Graphviz (e.g. something like dot -Tpng -odfa.png dfa.dot). --dfa-minimization Internal option: DFA minimization algorithm used by re2py. The moore option is the Moore algorithm (it is the default). The table option is the "table filling" algorithm. Both algorithms should produce the same DFA up to states relabeling; table filling is simpler and much slower and serves as a reference implementation. --eager-skip Internal option: make the generated lexer advance the input position eagerly -- immediately after reading the input symbol. This changes the default behavior when the input position is advanced lazily -- after transition to the next state. --no-lookahead Internal option, deprecated. It used to enable TDFA(0) algorithm. Unlike TDFA(1), TDFA(0) algorithm does not use one-symbol lookahead. It applies register operations to the incoming transitions rather than the outgoing ones. Benchmarks showed that TDFA(0) algorithm is less efficient than TDFA(1). --no-optimize-tags Internal option: suppress optimization of tag variables (useful for debugging). --posix-closure Internal option: specify shortest-path algorithm used for the construction of epsilon-closure with POSIX disambiguation semantics: gor1 (the default) stands for Goldberg-Radzik algorithm, and gtop stands for "global topological order" algorithm. --posix-prectable Internal option: specify the algorithm used to compute POSIX precedence table. The complex algorithm computes precedence table in one traversal of tag history tree and has quadratic complexity in the number of TNFA states; it is the default. The naive algorithm has worst-case cubic complexity in the number of TNFA states, but it is much simpler than complex and may be slightly faster in non-pathological cases. --stadfa Internal option, deprecated. It used to enable staDFA algorithm, which differs from TDFA in that register operations are placed in states rather than on transitions. Benchmarks showed that staDFA algorithm is less efficient than TDFA. --fixed-tags Internal option: specify whether the fixed-tag optimization should be applied to all tags (all), none of them (none), or only those in toplevel concatenation (toplevel). The default is all. "Fixed" tags are those that are located within a fixed distance to some other tag (called "base"). In such cases only the base tag needs to be tracked, and the value of the fixed tag can be computed as the value of the base tag plus a static offset. For tags that are under alternative or repetition it is also necessary to check if the base tag has a no-match value (in that case fixed tag should also be set to no-match, disregarding the offset). For tags in top-level concatenation the check is not needed, because they always match. WARNINGS -W Turn on all warnings. -Werror Turn warnings into errors. Note that this option alone doesn't turn on any warnings; it only affects those warnings that have been turned on so far or will be turned on later. -W Turn on warning. -Wno- Turn off warning. -Werror- Turn on warning and treat it as an error (this implies -W). -Wno-error- Don't treat this particular warning as an error. This doesn't turn off the warning itself. -Wcondition-order Warn if the generated program makes implicit assumptions about condition numbering. One should use either --header option or conditions block to generate a mapping of condition names to numbers and then use the autogenerated condition names. -Wempty-character-class Warn if a regular expression contains an empty character class. Trying to match an empty character class makes no sense: it should always fail. However, for backwards compatibility reasons re2py permits empty character classes and treats them as empty strings. Use the --empty- class option to change the default behavior. -Wmatch-empty-string Warn if a rule is nullable (matches an empty string). If the lexer runs in a loop and the empty match is unintentional, the lexer may unexpectedly hang in an infinite loop. -Wswapped-range Warn if the lower bound of a range is greater than its upper bound. The default behavior is to silently swap the range bounds. -Wundefined-control-flow Warn if some input strings cause undefined control flow in the lexer (the faulty patterns are reported). This is a dangerous and common mistake. It can be easily fixed by adding the default rule * which has the lowest priority, matches any code unit, and always consumes a single code unit. -Wunreachable-rules Warn about rules that are shadowed by other rules and will never match. -Wdeprecated-eof_rule Warn about standalone end of input rules $ that will be broken by the future changes and require fixing. At the moment these rules take precedence when conflicting with other rules, but after the introduction of generalized end of input symbol $ precedence order will change and these rules will become shadowed by other rules. -Wuseless-escape Warn if a symbol is escaped when it shouldn't be. By default, re2py silently ignores such escapes, but this may as well indicate a typo or an error in the escape sequence. -Wnondeterministic-tags Warn if a tag has n-th degree of nondeterminism, where n is greater than 1. -Wsentinel-in-midrule Warn if the sentinel symbol occurs in the middle of a rule --- this may cause reads past the end of buffer, crashes or memory corruption in the generated lexer. This warning is only applicable if the sentinel method of checking for the end of input is used. It is set to an error if re2c:sentinel configuration is used. -Wundefined-syntax-config Warn if the syntax file specified with --syntax option is missing definitions of some configurations. This helps to maintain user-defined syntax files: if a new release adds configurations, old syntax file will raise a warning, and the user will be notified. If some configurations are unused and do not need a definition, they should be explicitly set to .