/usr/bin/epp

{"severity_text":"INFO","timestamp":"2026-11-08T04:27:22.581280834Z","logger":"setup","caller":"runner/runner.go:243","body":"GIE build","service.name":"llm-d-epp","commit-sha":"b9393d1a9f1d5a72ecd86f8d2b643ab54c2a9b92","build-ref":"v1.0.0","severity_number":9}
Usage of /usr/bin/epp:
      --allow-experimental-plugins                     Allows loading of experimental Alpha plugins.
      --cert-path string                               Directory with tls.crt and tls.key for secure serving. Empty generates a self-signed certificate, which is only suitable for testing.
      --config-file string                             The path to the configuration file.
      --config-text string                             The configuration specified as text, in lieu of a file.
      --disable-endpoint-subset-filter                 Disables respecting the destination endpoint subset metadata for dispatching requests in EPP.
      --drain-timeout duration                         Graceful shutdown drain window. On SIGTERM the EPP goes NotServing and releases its leader lease immediately, then keeps serving ext_proc for this duration so in-flight and pre-DNS-refresh requests are not rejected. (default 30s)
      --emit-endpoint-scores                           Enables emitting the scheduler's per-endpoint scores in the request-path dynamic metadata under the 'x-gateway-destination-endpoint-scores' key of the 'envoy.lb' namespace.
      --enable-cert-reload                             Enables certificate reloading of the certificates specified in --cert-path.
      --enable-grpc-stream-metrics                     Enables ext_proc gRPC stream metrics (in-flight gauge, hold-duration histogram, completions counter by code).
      --enable-pprof                                   Enables pprof handlers. Defaults to true. Set to false to disable pprof handlers. (default true)
      --endpoint-selector string                       Selector to filter model server pods on. Supports Kubernetes label selector syntax: equality-based (e.g., 'app=vllm,env=prod'), set-based (e.g., 'env in (prod,staging),tier!=frontend'), and existence (e.g., 'key,!deprecated').
      --endpoint-target-ports ints                     Target ports of model server pods. Format: a comma-separated list of numbers without whitespace (e.g., '3000,3001,3002').
      --fairness-id-metric-label-limit int             Caps the number of distinct fairness_id label values recorded on metrics; values beyond the cap collapse to a single overflow series, and 0 collapses all of them. Bounds metric cardinality with many distinct fairness IDs. (default 1000)
      --feature-gates strings                          Comma-separated list of feature gates to enable or disable, in kubelet style (e.g., 'flowControl=true'). A bare name enables the gate. Applied after the configuration's featureGates list, so these override entries set there.
      --grpc-health-port uint16                        The port used for gRPC liveness and readiness probes. (default 9003)
      --grpc-max-recv-msg-size string                  Maximum size of a gRPC message to receive (e.g., 10MiB, 25MB).
      --grpc-max-send-msg-size string                  Maximum size of a gRPC message to send (e.g., 10MiB, 25MB).
      --grpc-port uint16                               gRPC port used for communicating with Envoy proxy. (default 9002)
      --ha-enable-leader-election                      Enables leader election for high availability. When enabled, readiness probes will only pass on the leader.
      --ha-lease-duration duration                     Leader election: how long a standby waits after the leader's last renewal before it takes the lease. (default 15s)
      --ha-lease-name string                           Leader election: name of the Lease in the pool namespace. Defaults to epp-<pool-namespace>-<pool-name>.llm-d.ai. A second EPP deployment for the same pool with its own lease can hold leadership while traffic moves to it.
      --ha-renew-deadline duration                     Leader election: how long the leader retries a failed lease renewal, for example against a slow API server, before it gives up leadership. Must be shorter than ha-lease-duration. (default 10s)
      --ha-retry-period duration                       Leader election: wait between attempts to acquire or renew the lease. (default 2s)
      --health-checking                                Enables health checking.
      --metrics-cert-dir string                        Directory with tls.crt and tls.key for the metrics endpoint. Empty serves metrics over plain HTTP. Independent of --secure-serving and --cert-path, which apply to the serving listener.
      --metrics-client-ca-file string                  PEM CA for metrics mTLS: require verified client certs.
      --metrics-endpoint-auth                          Enables authentication and authorization of the metrics endpoint. (default true)
      --metrics-port uint16                            The metrics port exposed by EPP. (default 9090)
      --metrics-staleness-threshold duration           Duration after which metrics are considered stale. This is used to determine if an endpoint's metrics are fresh enough. (default 2s)
      --plugin-state-staleness-threshold duration      Inactivity duration after which a request's plugin state (e.g. in-flight load accounting) is reaped. Set this above the maximum expected request duration to avoid releasing in-flight accounting for long-running requests. (default 5m0s)
      --pool-group inference.networking.k8s.io         Kubernetes resource group of the InferencePool this Endpoint Picker is associated with. Only inference.networking.k8s.io is currently supported. (default "inference.networking.k8s.io")
      --pool-name string                               Name of the InferencePool this Endpoint Picker is associated with.
      --pool-namespace string                          Namespace of the InferencePool this Endpoint Picker is associated with.
      --refresh-metrics-interval duration              Interval to refresh metrics. (default 50ms)
      --refresh-prometheus-metrics-interval duration   Interval to flush Prometheus metrics. (default 5s)
      --secure-serving                                 Serve the listener over TLS. (default true)
      --tls-cipher-suites strings                      Comma-separated list of TLS cipher suites for secure serving (Go crypto/tls names, e.g., TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256). Empty uses the crypto/tls default. Only effective for TLS 1.2 and below; TLS 1.3 cipher suites are not configurable.
      --tls-min-version string                         Minimum TLS version for secure serving (e.g., VersionTLS12, VersionTLS13). Empty uses VersionTLS12.
      --tracing                                        Enables emitting traces. (default true)
  -v, --v int                                          Number for the log level verbosity. (default 2)
      --zap-devel                                      Development Mode defaults(encoder=consoleEncoder,logLevel=Debug,stackTraceLevel=Warn). Production Mode defaults(encoder=jsonEncoder,logLevel=Info,stackTraceLevel=Error) (default true)
      --zap-encoder encoder                            Zap log encoding (one of 'json' or 'console')
      --zap-log-level level                            Zap Level to configure the verbosity of logging. Can be one of 'debug', 'info', 'error', 'panic' or any integer value > 0 which corresponds to custom debug levels of increasing verbosity
      --zap-stacktrace-level level                     Zap Level at and above which stacktraces are captured (one of 'info', 'error', 'panic').
      --zap-time-encoding time-encoding                Zap time encoding (one of 'epoch', 'millis', 'nano', 'iso8601', 'rfc3339' or 'rfc3339nano'). Defaults to 'epoch'.