LocalAI: A Self-Hosted OpenAI API for Every Kind of Model (GitHub, Scanned)

LocalAI is a drop-in replacement for the OpenAI API that you run yourself, and it goes well past text. Behind one endpoint on port 8080 it serves chat models, embeddings, rerankers, image and video generation, speech-to-text, text-to-speech and object detection, and it also speaks the Anthropic and ElevenLabs APIs. Each engine (llama.cpp, vLLM, whisper.cpp, diffusers, MLX and some 60 others) is a separate backend image pulled only when a model needs it, so the core stays small.

Ettore Di Giacinto started it in 2023 and it has grown into a platform: a web UI, multi-user accounts with API keys and quotas, built-in agents with MCP, and a distributed mode that spreads models across machines. It is MIT-licensed, written mostly in Go, and releases often; 4.10.0 came out ten days before this scan. No GPU is required to start.

• Repository: github.com/mudler/LocalAI
• Licence: MIT (MIT License)
• Language: Go. Stars: 49.3K. Forks: 4,473. Last push: Sep 27, 2026.
• Scan: safe, Sep 26, 2026, commit 92b8f1d

Who it is for

Self-hosters and teams who want one API for every kind of model, developers swapping OpenAI calls in an existing app for local ones, and anyone running on CPUs or a mix of hardware.

Getting started

1. Run the CPU image with Docker, then open http://localhost:8080 (AMD, Intel and Vulkan images are in the README)

docker run -ti --name local-ai -p 8080:8080 localai/localai:latest

2. Or on an NVIDIA GPU with CUDA 12

docker run -ti --name local-ai -p 8080:8080 --gpus all localai/localai:latest-gpu-nvidia-cuda-12

3. With the local-ai binary, pull and run a model from the gallery

local-ai run llama-3.2-1b-instruct:q4_k_m

The macOS app is an unsigned DMG, so the README has you clear the quarantine flag with sudo xattr -d com.apple.quarantine /Applications/LocalAI.app after installing. The localai.io site also offers a curl | sh installer; it downloads the release binary from GitHub, checks it against the published checksums, and uses sudo only to write to /usr/local/bin (LOCALAI_NO_SUDO=1 installs to ~/.local/bin instead).

Safety scan

We cloned mudler/LocalAI at commit 92b8f1d on Sep 26, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.

• No secrets, no self-decoding code and no committed binaries across 670,000 lines, mostly Go. The single bare-IP URL is a TEST-NET address (203.0.113.1) in a unit test.
• Three pattern hits, all about install commands rather than code that runs them: the Makefile fetches GoReleaser's runner with curl | bash for release builds, and the website's installer and its copy button both show curl -sSL https://localai.io/install.sh | sh. That script is 274 lines, verifies checksums and asks for sudo only to write the binary to /usr/local/bin.
• The 126 installer scripts are mostly per-backend install.sh and run.sh files that set up each Python or C++ engine; none pipes to a shell or edits your shell profile. The other sudo is .docker/install-base-deps.sh, which installs CUDA, Intel and Vulkan packages during the image build.
• 12 critical advisories after de-duplication. Eight are in go.mod: seven in golang.org/x/crypto 0.51.0, all about its SSH code, and one in gofiber/utils, an indirect dependency. The other four are torch 2.2.2 and 2.4.1, pinned by optional backends (Coqui TTS, faster-whisper, whisper-medusa); their torch.load advisory matters when loading untrusted model files. The React UI's lockfile has a single high entry.
• 36 workflows, none using pull_request_target, but all 70 third-party actions are pinned to tags rather than commits, in the pipeline that publishes the Docker images. Dependabot, Renovate, CodeQL and a security policy are all present.

What the scanner counted

The raw findings

Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.

By the numbers

Lines by language: Go 385.8K, JavaScript 69.3K, CSS 36.5K, Markdown 32.4K, YAML 31.6K, Python 31.2K.

Questions

Is LocalAI free?

Yes. LocalAI is MIT-licensed and free for any use, with no paid edition. Models from its gallery, Hugging Face or the Ollama registry are free to download under their own licences. Running it costs only your hardware, and since it runs on a CPU, a spare machine or a small server is enough to try it.

Does LocalAI need a GPU?

No. The default image runs on CPU, and small quantized models of 1B to 4B parameters are usable that way. A GPU makes it much faster, and LocalAI detects NVIDIA, AMD, Intel, Apple Silicon and Vulkan hardware and downloads the matching backend automatically. Image and video generation are where a GPU stops being optional in practice.

LocalAI or Ollama?

Ollama does one thing, text and vision models on llama.cpp, with the least setup. LocalAI is broader: speech, image, video and detection models behind the same API, OpenAI, Anthropic and ElevenLabs compatibility, multi-user accounts and a distributed mode, and it can run vLLM or llama.cpp as backends. Pick Ollama for a personal chat model and LocalAI for a shared, multi-modal server.

This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.