SGLang: Fast LLM Serving From the LMSYS Team (GitHub, Scanned)

SGLang is an inference server for running open models at scale. It loads language, vision-language and diffusion models from Hugging Face and serves them through an OpenAI-compatible API, using RadixAttention to reuse cached prompt prefixes between requests, plus continuous batching, speculative decoding, quantization and parallelism across many GPUs and machines. It is tuned for agent workloads with long shared prompts and for the generation step of reinforcement-learning training.

It is hosted by LMSYS, the group that started Chatbot Arena, and has about 38,000 stars and over 2,100 contributors, at v0.5.22. Hardware support is broad: NVIDIA from RTX cards to GB300, AMD Instinct, Google TPUs, Intel GPUs and Xeon CPUs, Huawei Ascend, and Apple Silicon. It is Apache-2.0.

Like vLLM, it is built for serving many requests rather than one person's chat. If you only want a model on your laptop, Ollama or LM Studio is simpler; if you are serving an application, SGLang and vLLM are the two to compare.

• Repository: github.com/sgl-project/sglang
• Licence: Apache-2.0 (Apache License 2.0)
• Language: Python. Stars: 38.2K. Forks: 9,898. Last push: Oct 26, 2026.
• Scan: safe, Oct 26, 2026, commit af21e2b

Who it is for

Teams serving open models to applications or many users, researchers running RL training that needs fast generation, and anyone with data-center GPUs who wants the most throughput from them.

Getting started

1. Install with uv in an activated Python environment (or docker pull lmsysorg/sglang:latest)

uv pip install --prerelease=allow sglang

2. Download a model from Hugging Face and serve it

sglang serve --model-path Qwen/Qwen3-8B

3. Query it like the OpenAI API (port 30000 by default)

curl http://localhost:30000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Hello"}]}'

The Cookbook at cookbook.sglang.io gives a tested launch command for each model and GPU. Pass --api-key before exposing the server beyond localhost.

Safety scan

We cloned sgl-project/sglang at commit af21e2b on Oct 26, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.

• No self-decoding code and no bare-IP URLs across 3.6 million lines, mostly Python with Rust. The one secret hit is a test private key in sgl-model-gateway/tests/security/mtls_test.rs.
• Of the 14 pattern hits, ten are Dockerfiles and CI scripts installing uv or Rust with their official curl | sh installers, one is a CI log-analysis script installing opencode the same way, and two are os.dup2 calls in mindspore_runner.py that redirect a worker's output to a log file, which the scanner mistook for a socket trick.
• python/setup.py has a custom build command that compiles the Rust extensions with cargo; it fetches nothing else. The one committed binary is a precompiled AMD GPU kernel (.co) for gfx950 chips, part of the attention kernels.
• 49 advisories, 2 critical: one in the zip crate (rust/sglang-radix-tree) and one in gRPC-Go (the Go bindings and their example server). The Python dependencies are ranges with no lockfile, so the check had nothing to read there.
• 123 workflows. Nine are listed under pull_request_target; those that actually run on it post status or apply labels, and the Cambricon MLU test is gated by maintainer labels and hands the commit to an isolated external runner without checking it out. Only 4 of 125 third-party actions are pinned to a commit. CodeQL and code of conduct present; no security policy.

What the scanner counted

The raw findings

Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.

By the numbers

Lines by language: Python 2.7M, Rust 306.4K, Markdown 179.9K, JSON 137K, JavaScript 118.1K, YAML 34.2K.

Questions

Is SGLang free?

Yes. SGLang is Apache-2.0, free for any use including commercial serving, with no paid edition. Your costs are the hardware or rented GPUs. Models are separate downloads, each under its own licence.

SGLang or vLLM?

Both are fast, open-source servers with OpenAI-compatible APIs and overlapping features, and the lead on benchmarks changes from release to release. SGLang's RadixAttention prefix cache helps most when many requests share long prompts, as in agents and multi-turn chat. Test both with your own model and traffic; switching is mostly a change of launch command.

What hardware does SGLang need?

Realistically Linux and a GPU: NVIDIA first, with AMD Instinct, Intel GPUs, Google TPUs and Huawei Ascend also supported, plus Xeon CPUs and Apple Silicon. Plan GPU memory for the full weights plus cache; an 8B model in 16-bit precision needs about 16 GB for its weights alone.

This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.