/usr/bin/ucx_perftest

/usr/bin/ucx_perftest: unrecognized option '--help'
  Usage: ucx_perftest [ server-address ] [ options ]

  Supported server-address:
     <hostname>         with default socket-based test setup
     <ip-address>       with default socket-based test setup
     lid:<dec|hex>      with MAD-based test setup (-K)
     guid:<hex>         with MAD-based test setup (-K)

  Common options:
     -t <test>      test to run:
           am_lat - UCT active message latency
          put_lat - UCT put latency
          add_lat - UCT atomic add latency
              get - UCT get latency / bandwidth / message rate
             fadd - UCT atomic fetch-and-add latency / rate
             swap - UCT atomic swap latency / rate
            cswap - UCT atomic compare-and-swap latency / rate
            am_bw - UCT active message bandwidth / message rate
           put_bw - UCT put bandwidth / message rate
           get_bw - UCT get bandwidth / message rate
           add_mr - UCT atomic add message rate
          tag_lat - UCP tag match latency
           tag_bw - UCP tag match bandwidth
     tag_sync_lat - UCP tag sync match latency
      tag_sync_bw - UCP tag sync match bandwidth
      ucp_put_lat - UCP put latency
       ucp_put_bw - UCP put bandwidth
          ucp_get - UCP get latency / bandwidth / message rate
          ucp_add - UCP atomic add bandwidth / message rate
         ucp_fadd - UCP atomic fetch-and-add latency / bandwidth / rate
         ucp_swap - UCP atomic swap latency / bandwidth / rate
        ucp_cswap - UCP atomic compare-and-swap latency / bandwidth / rate
        stream_bw - UCP stream bandwidth
       stream_lat - UCP stream latency
       ucp_am_lat - UCP am latency
        ucp_am_bw - UCP am bandwidth / message rate

     -a <send-device-type[:dev-id]>[,<recv-device-type[:dev-id]>]
                    Accelerator device type and device id to use for running the test.
                    device id is optional, it corresponds to the index of
                    the device in the list of available devices
     -L <level>     device cooperation level for gdaki (thread)
                    thread - thread level
                    warp   - warp level
                    block  - block level
                    grid   - grid level
     -F <size>      flow control window size for device tests (4).
                    This option defines the number of iterations per which a single flow control
                    request is sent.
     -Y <mode>      channel selection mode for device tests (single)
                    single          - use a single fixed channel (channel 0, default)
                    random[:<seed>] - use random channel per operation with optional random seed
                    per-thread      - use global thread ID
     -s <size>      list of scatter-gather sizes for single message (8)
                    for example: "-s 16,48,8192,8192,14"
                    compact form example: "-s 1024:16 expands to [1024, ..., 1024] with 16 elements
     -m <send mem type>[,<recv mem type>]
                    memory type of message for sender and receiver (host)
                        host - System memory
                        cuda - NVIDIA GPU memory
                        cuda-managed - NVIDIA GPU managed/unified memory
                        rocm - AMD/ROCm GPU memory
                        rocm-managed - AMD/ROCm GPU managed memory
                        rdma - RDMA device memory
                        ze-host - Intel/Ze USM host memory
                        ze-device - Intel/Ze GPU memory
                        ze-managed - Intel/Ze GPU managed memory
                        gaudi - HabanaLabs Gaudi memory
     -n <iters>     number of iterations to run (1000000)
     -w <iters>     number of warm-up iterations (10000)
     -c <cpulist>   set affinity to this CPU list (separated by comma) (off)
     -O <count>     maximal number of uncompleted outstanding sends (0)
     -i <offset>    distance between consecutive scatter-gather entries (0)
     -l             use loopback connection
                    in this case, the process will communicate with itself,
                    so passing server hostname is not allowed
     -o             do not progress the responder in one-sided tests
     -B             register memory with NONBLOCK flag
     -b <file>      read and execute tests from a batch file: every line in the
                    file is a test to run. The first word is a user-defined
                    test name, followed by command-line arguments, for example:

                        test_tag_bandwidth_64k -t tag_bw -s 65536 -n 10000

     -R <rank>      percentile rank of the percentile data in latency tests (50.0)
     -p <port>      TCP port to use for data exchange (13337)
     -6             Use IPv6 address for in data exchange
     -K <ca:port>   use MAD for test setup and synchronization
     -h             show this help message

  Output format:
     -N             use numeric formatting (thousands separator)
     -f             print only final numbers
     -v             print CSV-formatted output
     -X             print extra information about the operation
     -q             do not print error messages

  UCT only:
     -d <device>    device to use for testing
     -x <tl>        transport to use for testing
     -D <layout>    data layout for sender side:
                        short    - short messages (default, cannot be used for get)
                        shortiov - short io-vector messages (only for active messages)
                        bcopy    - copy-out (cannot be used for atomics)
                        zcopy    - zero-copy (cannot be used for atomics)
     -W <count>     flow control window size, for active messages (127)
     -H <size>      active message header size (8), included in message size
     -A <mode>      asynchronous progress mode (thread_spinlock)
                        thread_spinlock - separate progress thread with spin locking
                        thread_mutex - separate progress thread with mutex locking
                        signal - signal-based timer

  UCP only:
     -T <threads>[:<blocks>]
                    number of threads in the test (1).
                    blocks is optional, it corresponds to the number of device blocks
                    if blocks is specified, then threads value corresponds to the number
                    of device threads in each block
     -M <thread>    thread support level for progress engine (single)
                        single     - only the master thread can access
                        serialized - one thread can access at a time
                        multi      - multiple threads can access
     -D <layout>[,<layout>]
                    data layout for sender and receiver side (contig)
                        contig - Continuous datatype
                        iov    - Scatter-gather list
     -C             use wild-card tag for tag tests
     -U             force unexpected flow by using tag probe
     -r <mode>      receive mode for stream tests (recv)
                        recv       : Use ucp_stream_recv_nb
                        recv_data  : Use ucp_stream_recv_data_nb
     -I             create context with wakeup feature enabled
     -e             create endpoints with error handling support
     -E <mode>      wait mode for tests
                        poll       : repeatedly call worker_progress
                        sleep      : go to sleep after posting requests
     -H <size>      active message header size (0), not included in message size
     -y             do additional memcopy to the user memory in active message receive handler
     -z             pass pre-registered memory handle
     -g <IP>[:<port>], --daemon-local <IP>[:<port>]
                    IP address and port of the local daemon to offload UCP operations to
                    Port is optional, by default daemon port is (1338)
     -G <IP>[:<port>], --daemon-remote <IP>[:<port>]
                    IP address and port of the remote daemon to offload UCP operations to
                    Port is optional, by default daemon port is (1338)

   NOTE: When running UCP tests, transport and device should be specified by
         environment variables: UCX_TLS and UCX_[SELF|SHM|NET]_DEVICES.