Usage of /usr/bin/pd-sidecar: --cert-path string Directory with tls.crt and tls.key for secure serving. Empty generates a self-signed certificate, which is only suitable for testing. --configuration --configuration={port: 8085, model-server-port: 8203}. Inline configuration and file configuration are mutually exclusive. Sidecar configuration in YAML provided as inline specification. Example --configuration={port: 8085, model-server-port: 8203}. Inline configuration and file configuration are mutually exclusive. --configuration-file --configuration-file=/etc/config/sidecar-config.yaml Path to file which contains sidecar configuration in YAML. Example --configuration-file=/etc/config/sidecar-config.yaml. Inline configuration and file configuration are mutually exclusive. --data-parallel-size int the model server's data-parallel size (default 1) --decode-chunk-size int enables chunked decode mode when > 0; value is the token budget per chunk. For best performance should be a multiple of the block size. --ec-connector string the EC protocol between encoder and prefiller (for EPD mode). Supported: ec-example, ec-nixl. Leave empty to skip encoder stage. --enable-p2p-pull declare the OffloadingConnector P2P tier available for cached-prefix pulls when the PD connector is NIXL, i.e. engines run MultiConnector(NixlConnector + OffloadingConnector). Rejected with any other --kv-connector; offloading provides the tier natively without this flag. --enable-prefiller-sampling if true, the target prefill instance will be selected randomly from among the provided prefill host values --enable-ssrf-protection enable SSRF protection using InferencePool allowlisting --enable-tls strings stages to enable TLS for. Supported: prefiller, decoder, encoder. Can be specified multiple times or as comma-separated values. --inference-pool string InferencePool in namespace/name or name format (e.g., default/my-pool or my-pool). A single name implies the 'default' namespace. Can also use INFERENCE_POOL env var. --kv-connector string the KV protocol between prefiller and decoder. Supported: nixlv2, shared-storage, sglang, mooncake, offloading (default "nixlv2") --max-idle-conns-per-host int max idle keep-alive connections per host for reverse proxy transports; set to at least the expected concurrency (default 1024) --metrics-cert-dir string Directory with tls.crt and tls.key for the metrics endpoint. Empty serves metrics over plain HTTP. Independent of --secure-serving and --cert-path, which apply to the serving listener. --metrics-port int Port for the Prometheus /metrics endpoint (exposes the moriio_dns_* and llm_d_disagg_sidecar_* counters). 0 (the default) disables it. Takes precedence over the MORIIO_METRICS_ADDR env var. --model-server-port string the port the model server is listening on (default 8200) (default "8200") --mooncake-bootstrap-port int the port used to query the Mooncake bootstrap endpoint on prefill pods (only used with --kv-connector=mooncake) (default 8998) --moriio-decode-handshake-port int Base MoRI-IO handshake port on the decode pod. (default 6301) --moriio-decode-hosts strings Wide-EP: comma-separated decode-side pod hosts, emitted as the prefill request's remote_hosts. Kubernetes DNS names (e.g., 'pod-name.namespace.svc.cluster.local') are resolved to IPs at startup; literal IPs are used as-is. Pair with --moriio-dp-size-local. --moriio-decode-notify-port int Base MoRI-IO notify port on the decode pod. (default 61005) --moriio-dp-size int Data-parallel world size, emitted as kv_transfer_params[remote_dp_size] on both requests. Set to the engine DP size for Wide-EP (TP=1, DP>1); default 1 leaves the wire unchanged. (default 1) --moriio-dp-size-local int Wide-EP: per-pod DP size used to map a global DP rank to a pod index. Must satisfy --moriio-dp-size = dp-size-local * len(hosts). --moriio-local-pod-ip string Decode pod's routable address, used as the prefill request's remote_host. A Kubernetes DNS name (e.g., 'pod-name.namespace.svc.cluster.local') is resolved to an IP at startup; a literal IP is used as-is. Defaults to the POD_IP env var. Required with --moriio-write-mode. --moriio-parallel-decode-wait-timeout duration Backstop timeout for parallel dispatch: how long decode may wait on the prefill outcome/KV before being cancelled so the request fails instead of hanging. Only used with --moriio-parallel-dispatch. (default 30s) --moriio-parallel-dispatch Fire prefill and decode concurrently. OFF by default (serial dispatch via handleNIXLV2 is the SLA-safe path); opt-in only. Requires --moriio-write-mode. --moriio-prefill-handshake-port int Prefill pod's base MoRI-IO handshake port, used to build the decode request in parallel-dispatch mode. (default 6301) --moriio-prefill-notify-port int Prefill pod's base MoRI-IO notify port. (default 61005) --moriio-remote-hosts strings Wide-EP: comma-separated remote (prefill-side) pod hosts for per-DP-rank fan-out. Kubernetes DNS names (e.g., 'pod-name.namespace.svc.cluster.local') are resolved to IPs at startup; literal IPs are used as-is. Pair with --moriio-dp-size-local. --moriio-tp-size int Tensor-parallel size of the engines, echoed into kv_transfer_params[tp_size]. (default 1) --moriio-write-mode Enable MoRI-IO WRITE-mode passthrough in kv_transfer_params. Requires --kv-connector=nixlv2. --p2p-connector-port int the prefiller's OffloadingConnector P2P tier listening port, injected as remote_port on the decode request; with --data-parallel-size > 1 this is the rank-0 port and rank r uses port+r (used with --kv-connector=offloading or --enable-p2p-pull) (default 7777) --pool-group inference.networking.k8s.io Kubernetes resource group of the InferencePool this sidecar is associated with. Only inference.networking.k8s.io is currently supported. (default "inference.networking.k8s.io") --port string the port the sidecar is listening on (default "8000") --prefill-max-retries int max retry attempts when a prefill request fails with a 5xx error; 0 means no retries (default) --prefill-retry-backoff duration delay between prefill retry attempts (default 200ms) --secure-serving Serve the listener over TLS. (default true) --tls-cipher-suites strings Comma-separated list of TLS cipher suites for secure serving (Go crypto/tls names, e.g., TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256). Empty uses the crypto/tls default. Only effective for TLS 1.2 and below; TLS 1.3 cipher suites are not configurable. --tls-insecure-skip-verify strings stages to skip TLS verification for. Supported: prefiller, decoder, encoder. Can be specified multiple times or as comma-separated values. --tls-min-version string Minimum TLS version for secure serving (e.g., VersionTLS12, VersionTLS13). Empty uses VersionTLS12. --tracing Enable OpenTelemetry tracing --zap-devel Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error) --zap-encoder encoder Zap log encoding (one of 'json' or 'console') --zap-log-level level Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', 'panic' or any integer value > 0 which corresponds to custom debug levels of increasing verbosity --zap-stacktrace-level level Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic'). --zap-time-encoding time-encoding Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.