Skip to main content

Deploy with an agent

Copy agent prompt

Before you start

Connect your AWS account and choose a project. Review the upstream deployment instructions for the release you plan to pin. See upstream virtual key setup.

Data dependencies

PostgreSQL for virtual keys and database-backed spend tracking. A basic model proxy can run without SQL. See SQL databases for private writer access and connection details.

Deployment order

  1. If you need virtual keys, create private PostgreSQL first and wait for the writer. Obtain the application connection from Connect.
  2. Set DATABASE_URL to the writer for database-backed features. Set LITELLM_MASTER_KEY and provider credentials at runtime. Keep model routing in a versioned config.yaml that references environment variables for secrets.
  3. Deploy the pinned proxy image or a GitHub build containing the configuration. Use the configured HTTP port, commonly 4000, and restrict access to administrative routes.
  4. Start LiteLLM after its configured database is ready. Let the pinned image initialize its schema. Create scoped virtual keys only after the proxy is healthy.
Use Runtime variables for runtime credentials. For a Spring Winter web server, set an app-supported health path that returns HTTP 200 without credentials. Check History before running the application check.

Verify the deployment

Send a test completion through one approved model. If virtual keys are enabled, verify that a scoped key can use only its permitted models.