Ollama: Run LLMs on Your Own Machine (GitHub, Scanned)
Run open models on your own machine with one command, and talk to them over a local API.
Scanned 6 min read
Loading…
/** @github-tools */
Best Local AI Tools on GitHub
These projects run open AI models on your own computer, server or phone, so your prompts stay private and there is no per-message bill. Some are simple apps, others are engines and servers, and a few let you fine-tune models. Good ones support many model formats, keep up with new releases and have an active community. Pick by your hardware and comfort level: a desktop app for beginners, a server for teams.
24 repos, each cloned and scanned
// top 10, ranked
Updated 30 October 2026 · moves since 26 October 2026 · re-ranked every 3 days on what readers use, never sponsored
- 1
Ollama: Run LLMs on Your Own Machine (GitHub, Scanned)
no changeThe easiest way to download and run open models with one command, plus a local API that hundreds of other apps plug into.
- 2
llama.cpp: The Engine Under Local AI (GitHub, Scanned)
no changeThe fast C++ engine underneath most local AI apps, with its own server; best for tinkerers who want maximum speed and control.
- 3
Jan: An Offline ChatGPT Alternative for Your Desktop (GitHub, Scanned)
no changeA friendly desktop chat app that downloads and runs open models offline, a good first step for people who prefer not to use a terminal.
- 4
KoboldCpp: Run Local AI Models From One File, With a Web UI (GitHub, Scanned)
up 3One file that runs language, image, speech and music models with a built-in web UI, popular with storytellers and hobbyists.
- 5
vLLM: Fast LLM Serving on Your Own GPUs (GitHub, Scanned)
down 1A high-throughput inference server with an OpenAI compatible API, the usual choice for teams serving open models to many users on GPUs.
- 6
TextGen: Run Local LLMs with Every Knob Exposed (GitHub, Scanned)
up 2The long-running local LLM web app formerly called text-generation-webui, with several backends, chat, training and an API in one place.
- 7
Unsloth: Fine-Tune and Run LLMs Locally (GitHub, Scanned)
down 2Fine-tune open models faster on less GPU memory, then run them locally; suits anyone teaching a model their own data on modest hardware.
- 8
LocalAI: A Self-Hosted OpenAI API for Every Kind of Model (GitHub, Scanned)
down 2A self-hosted, OpenAI compatible API for text, image, voice and video models that runs with or without a GPU, handy as a drop-in replacement.
- 9
exo: Run Big AI Models Across Several Macs (GitHub, Scanned)
no changeJoins several Macs or PCs into one cluster so you can run models too big for any single machine.
- 10
Google AI Edge Gallery: Run AI Offline on Your Phone (GitHub, Scanned)
no changeGoogle's app for running Gemma and other open models fully offline on Android and iPhone, the simplest way to try local AI on a phone.
// all 24 in Run AI Locally
llama.cpp: The Engine Under Local AI (GitHub, Scanned)
The C++ inference engine underneath most local AI apps, with a CLI and an OpenAI-compatible server of its own.
Scanned 5 min read
vLLM: Fast LLM Serving on Your Own GPUs (GitHub, Scanned)
A high-throughput inference server for open models, with an OpenAI-compatible API.
Scanned 6 min read
Unsloth: Fine-Tune and Run LLMs Locally (GitHub, Scanned)
Fine-tune open models faster on less GPU memory, then run them locally in a desktop app or web UI.
Scanned 9 min read
LlamaFactory: Fine-Tune 100+ LLMs Without Writing Code (GitHub, Scanned)
Fine-tune Qwen, Llama, Gemma, DeepSeek and 100+ other models from a web UI or one command.
Scanned 3 min read
nanochat: Train Your Own ChatGPT for About $100 (GitHub, Scanned)
Andrej Karpathy's minimal full-stack recipe to train a small ChatGPT-style model and chat with it.
Scanned 5 min read
LocalAI: A Self-Hosted OpenAI API for Every Kind of Model (GitHub, Scanned)
A self-hosted, OpenAI-compatible API for text, image, voice and video models, with or without a GPU.
Scanned 9 min read
exo: Run Big AI Models Across Several Macs (GitHub, Scanned)
Joins several Macs or PCs into one cluster to run models too big for any single machine.
Scanned 6 min read
TextGen: Run Local LLMs with Every Knob Exposed (GitHub, Scanned)
The local LLM app formerly called text-generation-webui: five backends, chat, training and an API.
Scanned 5 min read
Jan: An Offline ChatGPT Alternative for Your Desktop (GitHub, Scanned)
A desktop chat app that downloads and runs open models offline, with an optional local API.
Scanned 6 min read
BitNet: Run 1-bit LLMs Fast on a CPU (GitHub, Scanned)
Microsoft's inference engine for 1-bit language models, built to run them fast on ordinary CPUs.
Scanned 3 min read
SGLang: Fast LLM Serving From the LMSYS Team (GitHub, Scanned)
A high-performance serving engine for LLMs and vision models, with an OpenAI-compatible API.
Scanned 6 min read
llamafile: An LLM in a Single Executable File (GitHub, Scanned)
Run a language model from one downloaded file on macOS, Windows, Linux or BSD, with nothing to install.
Scanned 4 min read
Google AI Edge Gallery: Run AI Offline on Your Phone (GitHub, Scanned)
Google's app for running Gemma and other open models fully offline on Android and iPhone.
Scanned 4 min read
gpt-oss: Run OpenAI's Open-Weight Models Yourself (GitHub, Scanned)
OpenAI's open-weight reasoning models, with reference code to run them on your own hardware.
Scanned 5 min read
WebLLM: Run Language Models Inside the Browser (GitHub, Scanned)
An engine that runs open language models entirely in the browser on WebGPU, with an OpenAI-style API.
Scanned 4 min read
MNN: Alibaba's Fast On-Device Engine for LLMs and AI Models (GitHub, Scanned)
Alibaba's lightweight inference engine for running LLMs and AI models on phones, PCs and devices.
Scanned 4 min read
Kohya's GUI: Train Your Own LoRA and Stable Diffusion Models (GitHub, Scanned)
A point-and-click interface for training LoRA, DreamBooth and fine-tuned image models on your GPU.
Scanned 5 min read
AI Toolkit: Train LoRAs for Image and Video Models (GitHub, Scanned)
A training suite with a web UI for fine-tuning image, video and audio models on your own GPU.
Scanned 5 min read
KoboldCpp: Run Local AI Models From One File, With a Web UI (GitHub, Scanned)
One file that runs local language, image, speech and music models with a built-in web UI and API.
Scanned 6 min read
PocketPal AI: Run Language Models Offline on Your Phone (GitHub, Scanned)
A private AI assistant for iPhone and Android that runs language models entirely on the phone.
Scanned 5 min read
Pinokio: One-Click Installer for Local AI Apps (GitHub, Scanned)
A desktop launcher that installs and runs local AI apps for images, video, voice and chat in one click.
Scanned 4 min read
MLX LM: Run and Fine-Tune LLMs on Apple Silicon (GitHub, Scanned)
Apple's MLX package for running, quantising and fine-tuning language models on a Mac.
Scanned 3 min read
Lemonade: A Local AI Server for Your GPU and NPU (GitHub, Scanned)
A local AI server for chat, code, speech and images on your own GPU or NPU, with OpenAI-style APIs.
Scanned 6 min read
// How we scan
Before a repository gets a post here we clone it and run the same checks on every one. The scanner is a script in this site's own repository, its findings are written out with file and line, and a person reads every hit before writing the verdict in the post. The last full run was on Oct 28, 2026.
- Pinned commit
- A fresh clone of the default branch. The commit and date are on every report, so the findings are about one snapshot, not the repository in general.
- Secrets
- Credential patterns with a known shape: cloud access keys, API tokens for OpenAI, Anthropic, Slack, GitHub, Google and Stripe, chat webhooks, private key blocks.
- Suspicious code
- Code that decodes a string and then executes it, pipes a download straight into a shell, opens a raw socket to a shell, reaches a known exfiltration service, mines cryptocurrency, or looks packed.
- Install-time code
- npm lifecycle scripts, setup.py command classes, Cargo build scripts, and installer shell scripts: whether they use sudo, edit your shell profile, or fetch and run remote code.
- Committed binaries
- Executables and compiled objects identified by their bytes, not their file extension, so a renamed binary still shows up.
- CI workflows
- GitHub Actions that run on pull_request_target and check out the pull request head (a classic way to leak secrets), write-all permissions, and third-party actions pinned to a tag instead of a commit.
- Network hosts
- Every host the source code talks to, so an unexpected one stands out.
- Known vulnerabilities
- Every pinned dependency in the repository's lockfiles looked up in OSV.dev, counted by severity. Includes development dependencies, which is why the counts are not zero for a large web app.
- Project hygiene
- A security policy, automated dependency updates, CodeQL, a licence, contributing guidelines.
- OpenSSF Scorecard
- The Open Source Security Foundation's independent score, where the project has one.
Safe means no credential patterns, no decode-and-execute code, no exfiltration hosts, no undocumented install-time behaviour, no risky workflow, and nothing in the findings a reviewer could not explain. Anything you should know before installing (an installer that pipes to a shell, a tool that reads a credential you already have, a licence with conditions, or a lockfile with unpatched advisories) is the note beside the badge.
Not covered: we do not run the software, audit its logic, or review commits made after the scan. The date and commit are on every report. Check them, and read the code yourself before you trust it with anything that matters.
// Questions
- Are these GitHub AI tools free to use?
- All of them are free to download, run and modify. The licences differ: most are MIT or Apache 2.0, ComfyUI is GPL-3.0, and Open WebUI and Dify ship their own licences with conditions on rebranding and multi-tenant hosting. Each post says which licence applies and what it asks of you.
- What does it mean that a repository was scanned?
- We cloned it at a specific commit and ran the checks listed under How we scan: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. Every hit was read by a person before the verdict was written, and the findings are in each post.
- Why do known vulnerabilities show up in a scan marked safe?
- Because a lockfile pins development dependencies too. A large web app carries thousands of packages and OSV knows of advisories in some of them; most affect build tooling and never reach a user. Each post shows the counts per lockfile and the worst entries so you can see where they sit, and the notes say whether any of it ships to you.
- How current is a scan?
- The date and commit are printed in every post. Active projects merge dozens of commits a week, so treat a report as a snapshot of that commit, not a standing guarantee, and re-check the repository before installing anything on a machine you care about.
- Which of these should I start with?
- To run a model on your own computer, install Ollama and add Open WebUI for a chat interface. For images and video, ComfyUI. To build an app or agent without much code, Dify. To let an agent write code or use a browser, OpenHands, Codex or browser-use. If you already use Claude Code or Codex, cost-xray shows what each session actually costs.