14:31:29 - LiteLLM:WARNING: get_model_cost_map.py:271 - LiteLLM: Failed to fetch remote model cost map from https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json: [Errno -3] Temporary failure in name resolution. Falling back to local backup.
Usage: litellm [OPTIONS]
Options:
--host TEXT Host for the server to listen on.
--port INTEGER Port to bind the server to.
--num_workers INTEGER Number of uvicorn / gunicorn workers to spin
up. Default is 1 (from
DEFAULT_NUM_WORKERS_LITELLM_PROXY)
--api_base TEXT API base URL.
--api_version TEXT For azure - pass in the api version.
-m, --model TEXT The model name to pass to litellm expects
--alias TEXT The alias for the model - use this to give a
litellm model name (e.g. "huggingface/codell
ama/CodeLlama-7b-Instruct-hf") a more user-
friendly name ("codellama")
--add_key TEXT The model name to pass to litellm expects
--headers TEXT headers for the API call
--save Save the model-specific config
--debug To debug the input
--detailed_debug To view detailed debug logs
--use_queue To use celery workers for async endpoints
--temperature FLOAT Set temperature for the model
--max_tokens INTEGER Set max tokens for the model
--request_timeout INTEGER Set timeout in seconds for completion calls
--drop_params Drop any unmapped params
--add_function_to_prompt If function passed but unsupported, pass it
as prompt
-c, --config TEXT Path to the proxy configuration file (e.g.
config.yaml). Usage `litellm --config
config.yaml`
--max_budget FLOAT Set max budget for API calls - works for
hosted models like OpenAI, TogetherAI,
Anthropic, etc.`
--telemetry BOOLEAN Helps us know if people are using this
feature. Turn this off by doing `--telemetry
False`
--log_config TEXT Path to the logging configuration file
--setup Run the interactive setup wizard to
configure providers and generate a config
file
-v, --version Print LiteLLM version
--health Make a chat/completions request to all llms
in config.yaml
--test proxy chat completions url to make a test
request to
--test_async Calls async endpoints /queue/requests and
/queue/response
--iam_token_db_auth Connects to RDS DB with IAM token
--num_requests INTEGER Number of requests to hit async endpoint
with
--run_gunicorn Starts proxy via gunicorn, instead of
uvicorn (better for managing multiple
workers)
--run_hypercorn Starts proxy via hypercorn, instead of
uvicorn (supports HTTP/2)
--ssl_keyfile_path TEXT Path to the SSL keyfile. Use this when you
want to provide SSL certificate when
starting proxy
--ssl_certfile_path TEXT Path to the SSL certfile. Use this when you
want to provide SSL certificate when
starting proxy
--ciphers TEXT Ciphers to use for the SSL setup.
--use_prisma_db_push Use prisma db push instead of prisma migrate
for database schema updates
--local for local debugging
--skip_server_startup Skip starting the server after setup (useful
for migrations only)
--keepalive_timeout INTEGER Set the uvicorn keepalive timeout in seconds
(uvicorn timeout_keep_alive parameter)
--timeout_worker_healthcheck INTEGER
Set the uvicorn worker health-check timeout
in seconds (uvicorn
timeout_worker_healthcheck parameter).
Requires uvicorn>=0.37.0. Only applies when
running uvicorn directly with
--num_workers>1; ignored under
--run_gunicorn / --run_hypercorn.
--max_requests_before_restart INTEGER
Restart worker after this many requests
(uvicorn: limit_max_requests, gunicorn:
max_requests)
--enforce_prisma_migration_check
Exit with error if database migration fails
on startup.
--use_v2_migration_resolver Opt into the v2 migration resolver. Avoids
the diff-and-force recovery path that can
cause schema thrashing during rolling
deploys where two LiteLLM versions contend
for the same DB. Default is the v1 resolver.
--reload Enable uvicorn hot reload (dev only).
Incompatible with --num_workers>1,
--run_gunicorn, and --run_hypercorn.
--help Show this message and exit.
Options (44)
| Option | Description | Source |
|---|---|---|
--host TEXT | Host for the server to listen on. | help |
--port INTEGER | Port to bind the server to. | help |
--num_workers INTEGER | Number of uvicorn / gunicorn workers to spin up. | help |
--api_base TEXT | API base URL. | help |
--api_version TEXT | For azure - pass in the api version. | help |
-m, --model TEXT | The model name to pass to litellm expects | help |
--alias TEXT | The alias for the model - use this to give a litellm model name (e.g. "huggingface/codell ama/CodeLlama-7b-Instruct-hf") a more user- friendly name ("codellama") | help |
--add_key TEXT | The model name to pass to litellm expects | help |
--headers TEXT | headers for the API call | help |
--save | Save the model-specific config | help |
--debug | To debug the input | help |
--detailed_debug | To view detailed debug logs | help |
--use_queue | To use celery workers for async endpoints | help |
--temperature FLOAT | Set temperature for the model | help |
--max_tokens INTEGER | Set max tokens for the model | help |
--request_timeout INTEGER | Set timeout in seconds for completion calls | help |
--drop_params | Drop any unmapped params | help |
--add_function_to_prompt | If function passed but unsupported, pass it as prompt | help |
-c, --config TEXT | Path to the proxy configuration file (e.g. config.yaml). | help |
--max_budget FLOAT | Set max budget for API calls - works for hosted models like OpenAI, TogetherAI, Anthropic, etc.` | help |
--telemetry BOOLEAN | Helps us know if people are using this feature. | help |
--log_config TEXT | Path to the logging configuration file | help |
--setup | Run the interactive setup wizard to configure providers and generate a config file | help |
-v, --version | Print LiteLLM version | help |
--health | Make a chat/completions request to all llms in config.yaml | help |
--test | proxy chat completions url to make a test request to | help |
--test_async | Calls async endpoints /queue/requests and /queue/response | help |
--iam_token_db_auth | Connects to RDS DB with IAM token | help |
--num_requests INTEGER | Number of requests to hit async endpoint with | help |
--run_gunicorn | Starts proxy via gunicorn, instead of uvicorn (better for managing multiple workers) | help |
--run_hypercorn | Starts proxy via hypercorn, instead of uvicorn (supports HTTP/2) | help |
--ssl_keyfile_path TEXT | Path to the SSL keyfile. | help |
--ssl_certfile_path TEXT | Path to the SSL certfile. | help |
--ciphers TEXT | Ciphers to use for the SSL setup. | help |
--use_prisma_db_push | Use prisma db push instead of prisma migrate for database schema updates | help |
--local | for local debugging | help |
--skip_server_startup | Skip starting the server after setup (useful for migrations only) | help |
--keepalive_timeout INTEGER | Set the uvicorn keepalive timeout in seconds (uvicorn timeout_keep_alive parameter) | help |
--timeout_worker_healthcheck INTEGER | Set the uvicorn worker health-check timeout in seconds (uvicorn timeout_worker_healthcheck parameter). | help |
--max_requests_before_restart INTEGER | Restart worker after this many requests (uvicorn: limit_max_requests, gunicorn: max_requests) | help |
--enforce_prisma_migration_check | Exit with error if database migration fails on startup. | help |
--use_v2_migration_resolver | Opt into the v2 migration resolver. | help |
--reload | Enable uvicorn hot reload (dev only). | help |
--help | Show this message and exit. | help |