vLLM: Fast LLM Serving on Your Own GPUs (GitHub, Scanned)

vLLM is what you run when a model has to serve more than one person. It loads open models from Hugging Face onto one GPU or many and serves them through an OpenAI-compatible API, plus Anthropic's Messages format and gRPC, getting far more requests per second out of the same card than a desktop runner does. The idea that started it, PagedAttention, manages the attention cache in pages the way an operating system manages memory; continuous batching, prefix caching, speculative decoding and a long list of quantization formats came after.

It began in UC Berkeley's Sky Computing Lab in 2023 and is now one of the largest open-source AI projects on GitHub: over 3,500 contributors, 22,000 commits and 106 releases, the latest 0.30.0. It is Apache-2.0, supports more than 200 model architectures, and runs on NVIDIA, AMD and Intel GPUs and x86, ARM and PowerPC CPUs, with plugins for TPUs, Gaudi, Ascend and Apple Silicon. It is also what the other tools on this list reach for when a single-user runner is not enough: LocalAI ships it as a backend.

• Repository: github.com/vllm-project/vllm
• Licence: Apache-2.0 (Apache License 2.0)
• Language: Python. Stars: 92.7K. Forks: 22.7K. Last push: Sep 27, 2026.
• Scan: safe, Sep 27, 2026, commit a9eafde

Who it is for

Teams serving a model to an application or many users, anyone with a data-center or large NVIDIA or AMD GPU who needs throughput, and developers who want a self-hosted OpenAI-compatible endpoint that holds up under load.

Getting started

1. Create an environment and install with uv (Linux, Python 3.10 to 3.13; pip install vllm also works)

uv venv --python 3.12 --seed && source .venv/bin/activate && uv pip install vllm --torch-backend=auto

2. Download a model from Hugging Face and serve it on localhost:8000

vllm serve Qwen/Qwen2.5-1.5B-Instruct

3. Query it like the OpenAI API

curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"Qwen/Qwen2.5-1.5B-Instruct","messages":[{"role":"user","content":"Who won the world series in 2020?"}]}'

The server accepts requests from anyone who can reach it unless you pass --api-key (or set VLLM_API_KEY). Set one before binding it to anything but localhost.

Safety scan

We cloned vllm-project/vllm at commit a9eafde on Sep 27, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.

• No secrets, no self-decoding code, no committed binaries and no bare-IP URLs across 2.2 million lines, most of them Python with CUDA, C++ and a new Rust front end.
• Four pattern hits, none in what you install. Three are build scripts fetching toolchains with curl | sh (uv in the main Dockerfile, rustup in the PowerPC build script and tools/build_rust.sh); the fourth is a kernel benchmark redirecting stderr to /dev/null, which the scanner mistook for a socket trick.
• setup.py has a custom build command, which is how a CUDA project compiles its kernels. With VLLM_USE_PRECOMPILED set it downloads prebuilt kernels from vLLM's own wheels.vllm.ai and asks api.github.com for the latest main commit; a normal pip or uv install gets a finished wheel and runs none of it. The 36 installer scripts are CI jobs (28 under .buildkite) and optional kernel installers under tools/, and none uses sudo or pipes to a shell.
• The only lockfile is rust/Cargo.lock: 26 advisories, none critical, two high (an out-of-bounds read in PyO3's list iterators and a denial of service in rustls-webpki's revocation-list parsing), the rest mostly unmaintained-crate notices. The Python dependencies are version ranges in requirements files with no lockfile, so the advisory check had nothing to read there.
• 12 workflows. Three use pull_request_target (bots for automerge labels, new pull requests and CI notifications) and none checks out the pull request; all 7 third-party actions are pinned to a commit. Dependabot, security policy, contributing guide and code of conduct present.

What the scanner counted

The raw findings

Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.

By the numbers

Lines by language: Python 1.7M, JSON 161.1K, Rust 126.3K, CUDA 55K, Markdown 49K, C/C++ header 33.3K.

Questions

Is vLLM free?

Yes. vLLM is Apache-2.0, free for any use including commercial serving, and there is no paid edition of the software. Your costs are the hardware, or the rented cloud GPUs, it runs on. Models are separate downloads from Hugging Face, each under its own licence, and some families such as Llama and Gemma carry use conditions of their own.

vLLM or Ollama?

Ollama is built for one person on one machine: a simple install, quantized GGUF models, and it runs on a laptop. vLLM is built for throughput: many concurrent requests batched on a GPU, full-precision or FP8 models, and parallelism across several GPUs. If you are chatting with a model yourself, use Ollama. If an application or a team sends it traffic, vLLM serves far more requests from the same card.

What hardware does vLLM need?

Linux and a GPU, realistically. It targets NVIDIA cards first, also supports AMD ROCm, Intel GPUs and x86, ARM and PowerPC CPUs, and reaches TPUs, Gaudi and Ascend through plugins. On a Mac it runs through the separate vLLM-Metal project. Plan GPU memory for full weights plus cache: a 7B model in 16-bit precision needs about 15 GB for its weights alone.

This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.