LiteLLM v1 AI Gateway with Spend Tracking

LiteLLM is an open-source AI gateway that puts OpenAI, Anthropic, Gemini, Bedrock and 100+ other providers behind one OpenAI-compatible API. This template runs the LiteLLM proxy v1.104.2 with Postgres for keys and spend tracking and Redis for caching and shared rate limits.

What you get

  • The official litellm-database image v1.104.2 with an Admin UI at /ui
  • Postgres 18 (volume) for models, virtual keys, teams, budgets and spend logs
  • Redis 8.2 (volume) for the response cache and shared rate-limit and router state
  • Generated master key, salt key and Admin UI password
  • Models managed from the Admin UI; optional provider key variables for OpenAI, Anthropic, Gemini and OpenRouter

Good for

  • Giving each team or app its own virtual key with a budget, model allow-list and RPM/TPM limits
  • Switching between providers or self-hosted models without changing application code
  • Tracking LLM spend per key, user and team
  • Offering one internal OpenAI-compatible endpoint to tools such as n8n or Open WebUI

Services in this template

  • Redis · image redis:8.2
  • LiteLLM · from baranberkay96/litellm-railway
  • Postgres · image ghcr.io/railwayapp-templates/postgres-ssl:18

Questions people ask

How do I log in?

Open https://{your-domain}/ui and sign in as admin with the generated UI_PASSWORD from the LiteLLM service variables. Keep LITELLM_MASTER_KEY for administration only and create virtual keys for applications.

Do I need a provider API key?

Yes, at least one. Paste it when adding a model in the Admin UI, or set an optional variable such as OPENAI_API_KEY or ANTHROPIC_API_KEY. Provider usage is billed by the provider.

Is the Redis cache on for every request?

No. By default caching is opt-in per request (LITELLM_CACHE_MODE=default_off). Send a cache use-cache flag in the request, or set the variable to default_on.

How do I upgrade or scale?

The version is the FROM line in services/litellm/Dockerfile; change the tag and redeploy, and migrations run on start. For more throughput raise NUM_WORKERS or add replicas; Redis keeps rate limits shared. Never change LITELLM_SALT_KEY after adding models.

What licence applies?

The NOTICE file lists LiteLLM as MIT outside its enterprise/ directory, which has a separate enterprise licence. This template does not enable enterprise features.

Full template README

Deploy and Host LiteLLM with Railway

LiteLLM is an open-source AI gateway that puts OpenAI, Anthropic, Gemini, Bedrock, Azure, OpenRouter and 100+ other model providers behind one OpenAI-compatible API. This community template runs the LiteLLM proxy with Postgres for keys and spend tracking and Redis for caching and shared rate limits.

About Hosting LiteLLM

The LiteLLM proxy is a Python service that translates OpenAI-style requests to each provider's API, then records who called which model and what it cost. Running it as a shared gateway needs a database for virtual keys, teams, budgets and spend logs, and a shared cache so rate limits and cooldowns stay consistent when the proxy runs several processes or replicas. This template deploys the official LiteLLM database image with a small production configuration: models are managed from the Admin UI, spend is written in batches, the Redis cache and router state are enabled, and the master key, salt key and Admin UI password are generated for you at deploy time.

Common Use Cases

  • Give every team or application its own virtual key with a monthly budget, model allow-list and RPM/TPM limits.
  • Switch between OpenAI, Anthropic, Gemini or self-hosted models without changing application code.
  • Load-balance and fail over across several deployments or API keys of the same model.
  • Track LLM spend per key, user and team, and export it to observability tools such as Langfuse.
  • Offer one internal OpenAI-compatible endpoint to tools like n8n, Open WebUI or coding assistants.

Dependencies for LiteLLM Hosting

  • LiteLLM proxy image ghcr.io/berriai/litellm-database:v1.104.2
  • PostgreSQL (Railway Postgres 18)
  • Redis (Railway Redis 8.2)
  • At least one LLM provider API key (added in the Admin UI or as an optional variable)

Deployment Dependencies

Implementation Details

Service Role Public Storage
LiteLLM OpenAI-compatible API, Admin UI (/ui) Yes (port 4000) –
Postgres Models, keys, teams, budgets, spend logs No Volume
Redis Response cache, rate-limit and router state No Volume

First login

  1. Wait for the LiteLLM healthcheck (/health/readiness) to pass.
  2. Open https://{your-domain}/ui and sign in with UI_USERNAME (admin) and the generated UI_PASSWORD from the LiteLLM service variables.
  3. Add a model under Models (paste the provider key there, or set the optional OPENAI_API_KEY / ANTHROPIC_API_KEY / GEMINI_API_KEY / OPENROUTER_API_KEY variables).
  4. Create a virtual key and call https://{your-domain}/v1/chat/completions with it. Keep LITELLM_MASTER_KEY for administration only.

Caching: the Redis cache is opt-in per request by default (LITELLM_CACHE_MODE=default_off); send "cache": {"use-cache": true} or set the variable to default_on.

Scaling: increase NUM_WORKERS for more processes in one container, or add replicas to the LiteLLM service; Redis keeps rate limits consistent across them. Each worker uses up to 10 Postgres connections.

Pinning and upgrades: the version is set in services/litellm/Dockerfile (FROM ghcr.io/berriai/litellm-database:v1.104.2). Change the tag to a newer stable release and redeploy; database migrations run automatically on start. Never change LITELLM_SALT_KEY after models have been added.

Why Deploy LiteLLM on Railway?

Railway gives the gateway a public HTTPS domain, private networking to its Postgres and Redis, generated secrets and usage-based billing, and other services in the same project can reach it privately. Scaling is a replica or worker setting, and upgrades are a one-line tag change.

Guides for this template

View LiteLLM v1 AI Gateway with Spend Tracking on railway.com

Similar templates

TemplateCategoryDeploymentsDeploys
9Router
Self-hosted AI router for Claude, Gemini, Cursor & OpenAI tools.
AI/ML9,734
flowise-railway
Flowise - low-code LLM apps builder
AI/ML8,512
LibreChat
Open-source ChatGPT clone: multi-model, agents, search, multi-user auth
AI/ML6,762
Hermes Agent by Nous Research
Template for Hermes Agent by Nous Research
AI/ML6,413
OpenClaw
[Homebrew support] The AI that actually does things
AI/ML5,167
Hermes Agent
Self Host Hermes Agent. Autonomous AI agent w/ self-improving, tools & more
AI/ML4,092