WebLLM: Run Language Models Inside the Browser (GitHub, Scanned)

WebLLM runs a language model inside a web page. The model downloads once into the browser's cache and then runs on the visitor's own GPU through WebGPU, with no server doing inference and no prompt leaving the device. For developers it looks like the OpenAI API: create an engine with a model name, then call engine.chat.completions.create with messages, with streaming, JSON mode and seeding supported.

It is on this list because it removes the server from private AI features. A website, a Chrome extension or an offline web app can offer chat, summarization or structured extraction at zero inference cost to the publisher. It supports Llama, Phi, Gemma, Mistral and Qwen models compiled for the MLC runtime, runs in web workers or service workers so the page stays responsive, and can be imported from npm or straight from a CDN.

WebLLM comes from the MLC AI project, which also makes MLC LLM, and has about 19,000 stars. It is Apache-2.0 licensed, and a paper describing it is on arXiv. WebLLM Chat, a ready-made chat app built on it, runs at chat.webllm.ai.

• Repository: github.com/mlc-ai/web-llm
• Licence: Apache-2.0 (Apache License 2.0)
• Language: TypeScript. Stars: 19.2K. Forks: 1,388. Last push: Oct 3, 2026.
• Scan: safe, Oct 3, 2026, commit 7f0f0d1

Who it is for

Web developers who want AI features without paying for inference or handling user data, extension authors, and anyone curious to try a local model with nothing to install.

Getting started

1. Try it with no code in a WebGPU browser such as recent Chrome or Edge

open https://chat.webllm.ai

2. Add the package to your web project

npm install @mlc-ai/web-llm

3. Load a model and chat with it, OpenAI style

import { CreateMLCEngine } from "@mlc-ai/web-llm";
const engine = await CreateMLCEngine("Llama-3.1-8B-Instruct-q4f32_1-MLC");
const reply = await engine.chat.completions.create({ messages: [{ role: "user", content: "Hello!" }] });
console.log(reply.choices[0].message.content);

The first load downloads the whole model, several gigabytes for an 8B model, into browser storage, and it needs a GPU with enough memory. Smaller models such as Qwen2 0.5B or 1.5B load much faster and suit phones and older laptops.

Safety scan

We cloned mlc-ai/web-llm at commit 7f0f0d1 on Oct 3, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.

• No secrets, no bare-IP URLs and no committed binaries across 326 files and about 45,000 lines, 37,000 of them TypeScript. The one pattern hit is a long line in tests/function_calling.test.ts, a test fixture. The only npm hook installs Husky for contributors.
• Network hosts are Hugging Face for model weights (over 200 references), GitHub for compiled model libraries, and MLC's own sites; we found no analytics. Prompts and replies stay in the browser.
• 9 known advisories, none critical (7 high), in package-lock.json (475 packages): js-yaml, brace-expansion and braces, used by linting, testing and build tools rather than by the engine at run time.
• 5 workflows, none using pull_request_target; the one third-party action is not pinned to a commit. Security policy, CodeQL, licence and contributing guide present; no Dependabot.

What the scanner counted

The raw findings

Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.

By the numbers

Lines by language: TypeScript 37.4K, JavaScript 2,576, Markdown 1,414, CSS 1,368, HTML 978, JSON 867.

Questions

Is WebLLM free?

Yes. WebLLM is Apache-2.0 and free, and because inference happens on each user's device, there is no per-request cost for you or them. The models it runs are free to download, each under its own licence, such as Meta's Llama licence or Apache-2.0 for Qwen.

Which browsers support WebLLM?

Any browser with WebGPU enabled. Recent desktop Chrome and Edge have it on by default, and support in Safari and Firefox has been arriving in recent versions. If WebGPU is missing, the engine reports an error rather than falling back to the CPU.

Does WebLLM send my prompts to a server?

No. Model weights are downloaded from Hugging Face or the configured host, but prompts and replies are processed entirely in the browser on your own hardware.

This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.