MrTechKing

6 min read

MiniMind: Train Your Own Small LLM From Scratch (GitHub, Scanned)

A free project that teaches you to train a 64M-parameter language model from scratch, in about 2 hours on one RTX 3090.

MiniMind logo
✅
Scan: safe. Nothing malicious. Two things to know: the requirements file pins old packages with known advisories, mostly ones no script imports, and the .pth checkpoints load with torch.load, so only use weights from the project or your own training. Scanned Nov 21, 2026; the full report is below.

MiniMind is a free, open project that shows you how to build a small language model yourself, from the tokenizer up. Its scripts train minimind-3, a 64-million-parameter chat model about 1/2,700 the size of GPT-3, through pretraining and instruction tuning, and the README estimates about 2.3 hours and 3 yuan (under half a US dollar) of rented RTX 3090 time for those two steps. In your first ten minutes you can skip training altogether: download the finished model and chat with it in the terminal.

It earns its place because the core pieces are written by hand in plain PyTorch instead of hidden behind a library call: the model itself (a normal version and a mixture-of-experts one), LoRA, DPO, PPO and GRPO reinforcement learning, distillation, tool calling and a thinking mode. The datasets for every stage are published on ModelScope and Hugging Face, and the finished models load in transformers, llama.cpp, vLLM and Ollama. A small Streamlit chat page and an OpenAI-compatible API server let you try your own model in a chat window or plug it into Open WebUI. The main README is in Chinese, with a full English version in README_en.md.

MiniMind is Apache-2.0 licensed and maintained by the developer jingyaogong, with about 67,800 stars and 30 contributors since July 2024. GitHub releases are rare: the latest, v2, is from 21 October 2025. The current model, minimind-3, came out on 1 April 2026, and the code was last updated on 18 November 2026.

  • Repository: github.com/jingyaogong/minimind
  • Licence: Apache-2.0 (Apache License 2.0)
  • Language: Python. Stars: 67.8K. Forks: 8,863. Last push: Nov 18, 2026.
  • Scan: safe, Nov 21, 2026, commit cc9629e

Who it is for

Students and developers who want to see how an LLM works inside, people moving into AI work who want hands-on training experience, and teachers looking for a course project. Chatting with the finished model works on an ordinary CPU; training needs Python, some PyTorch and, in practice, an NVIDIA GPU.

Getting started

1. Install PyTorch for your machine first (see pytorch.org), then get the code and its packages

git clone --depth 1 https://github.com/jingyaogong/minimind && cd minimind && pip install -r requirements.txt

2. Download the finished minimind-3 model

modelscope download --model gongjy/minimind-3 --local_dir ./minimind-3

3. Chat with it in the terminal

python eval_llm.py --load_from ./minimind-3

4. Or, if you already use Ollama, try it there

ollama run jingyaogong/minimind-3

5. To train your own, put pretrain_t2t_mini.jsonl and sft_t2t_mini.jsonl from the dataset page into ./dataset, then run

cd trainer && python train_pretrain.py && python train_full_sft.py

6. Then, back in the main folder, chat with the model you trained

python eval_llm.py --weight full_sft

requirements.txt pins exact, older versions of about 30 packages, so install them in a fresh virtual environment, not your main Python. The README's own install line adds a package mirror in China (mirrors.aliyun.com); we left it out. Models and datasets download from ModelScope or Hugging Face. Our scan covers the repository's code, not the model weights, the datasets or the PyTorch you install.

Safety scan

We cloned jingyaogong/minimind at commit cc9629e on Nov 21, 2026 and ran the checks described on the GitHub Tools page: credential patterns, decode-and-execute code, install-time scripts, committed binaries, risky CI workflows, every host the code talks to, known vulnerabilities in pinned dependencies, and project hygiene. A person read every hit. This is what we found.

  • 0 secret hits and no committed binaries across 49 files and about 40,400 lines, most of it model/tokenizer.json, the tokenizer's vocabulary. The rest is Python scripts, the two READMEs and the diagrams in images/.
  • 1 pattern hit: a very long line at trainer/train_tokenizer.py line 121. It is the chat template, a 4,085-character text string the script writes into tokenizer_config.json, not hidden code. We also read the calculator the tool-calling demos use, in trainer/trainer_utils.py: it parses arithmetic with Python's ast module and allows only listed math functions, instead of calling eval.
  • No install hooks or install scripts. Setup is git clone plus pip install -r requirements.txt, and PyTorch is left for you to install. The chat and training scripts load .pth checkpoints with torch.load, which can run code hidden in a tampered file on PyTorch versions before 2.6, so load only weights from the project or your own training. The released minimind-3 model is a safetensors file with no custom code in its folder.
  • 98 advisories, 6 critical and 46 high, all from the exact versions pinned in requirements.txt. The 6 critical ones are in nltk 3.8 (5) and sentence-transformers 2.3.1 (1), and most of the high ones are in nltk too; no MiniMind script imports either package, nor ujson, Flask-CORS or sentencepiece, which hold most of the other high ones. The packages the scripts do use add 4 high advisories in transformers 4.57.6, all about loading a malicious model folder from someone else, plus moderate ones in jinja2, datasets and Streamlit.
  • No workflows, so no pull_request_target. The hosts in the code are GitHub and ModelScope. Training charts go to SwanLab, a cloud dashboard like WandB, only when you add --use_wandb; it is off by default. The optional Streamlit chat page sends Streamlit's own usage statistics unless you turn them off, and scripts/serve_openai_api.py listens on all network interfaces (port 8998) with no password. Code of conduct present; no security policy, Dependabot or CodeQL.

What the scanner counted

CheckResult
SecretsNone found.
Suspicious code1 pattern hit found and read; every one is listed under the raw findings.
Install-time codeNone: nothing runs at install beyond the package manager itself.
Committed binariesNone.
CI workflowsNo GitHub Actions workflows.
Network hosts2 distinct hosts referenced from source; most often github.com, www.modelscope.cn. No URLs to bare IP addresses.
Known vulnerabilities98 advisories across 31 pinned packages: 6 critical, 46 high, 41 moderate, 3 low, 2 unrated. requirements.txt: 32 packages, 98 advisories.
Project hygieneHas licence file. Missing security policy, automated dependency updates, CodeQL, contributing guide.
OpenSSF ScorecardNot scored: the project is not in Scorecard's weekly index.

The raw findings

Every hit the scanner wrote out, with a link to the exact line at the scanned commit. Secrets candidates are redacted.

Pattern hits (1)
WhereRuleMatch
trainer/train_tokenizer.py:121very-long-line4085 chars
Worst known vulnerabilities (24 of 98)
AdvisorySeverityPackageSummary
GHSA-7p94-766c-hgjpcriticalnltk@3.8NLTK has a Zip Slip Vulnerability
GHSA-m4rf-3fr8-xwx3criticalnltk@3.8NLTK: JVM argument injection bypass via per-call options in the NLTK Stanford wrappers (incomplete fix of CVE-2026-12841…
GHSA-rhp5-r9x4-f5g2criticalnltk@3.8NLTK: Unsafe Pickle Deserialization in TransitionParser Allows Remote Code Execution
GHSA-x99w-6fgc-pmfwcriticalnltk@3.8NLTK: Allowlisted pickle loaders still permit code execution in current source
CVE-2026-0848criticalnltk@3.8
GHSA-jhr6-gm9c-rqjvcriticalsentence_transformers@2.3.1sentence-transformers local model loading bypasses trust_remote_code and executes custom Python
GHSA-hxwh-jpp2-84pmhighflask_cors@4.0.0Flask-CORS allows the `Access-Control-Allow-Private-Network` CORS header to be set to true by default
GHSA-3gq4-3j92-5w49highnltk@3.8NLTK: Corpus Reader Sandbox Bypass
GHSA-469j-vmhf-r6v7highnltk@3.8NLTK has a Downloader Path Traversal Vulnerability (AFO) - Arbitrary File Overwrite
GHSA-568f-pv23-39p4highnltk@3.8NLTK: Stable FrameNet and NKJP readers parse outside-root XML
GHSA-68j8-pq59-fqgmhighnltk@3.8NLTK has a Path Traversal issue
GHSA-6hm5-jgcp-p838highnltk@3.8Natural Language Toolkit (NLTK): Path Traversal in NKJPCorpusReader leads to Arbitrary File Read and bypasses the nltk.p…
GHSA-6hwm-xvph-95vmhighnltk@3.8NLTK: Uncontrolled search path when invoking the Graphviz 'dot' binary
GHSA-6ww7-3frv-cqxhhighnltk@3.8NLTK: pathsec SSRF protection can be bypassed when a proxy is configured
GHSA-7mxv-7h3q-9324highnltk@3.8NLTK dot2img runs Graphviz 'dot' without the trusted-exec chokepoint, enabling local code execution via a writable PATH …
GHSA-848c-c2cx-j7qxhighnltk@3.8NLTK vulnerable to Eval Injection via collocations CLI arguments
GHSA-8846-p9w9-5frfhighnltk@3.8Natural Language Toolkit (NLTK): Uncontrolled recursion (crash DoS) in Synset.tree()/acyclic_tree() via WordNet relation…
GHSA-89p3-fcch-88phhighnltk@3.8NLTK: Quadratic-time CPU denial of service in CCG lexicon parser (fromstring) on wide bracket bodies and operator chains
GHSA-8fx7-8jr8-84rvhighnltk@3.8NLTK: Unbounded CPU consumption in SteppingRecursiveDescentParser.parse() with ambiguous or self-embedding grammars
GHSA-8mgp-746c-j5xphighnltk@3.8NLTK: Model-artifact APIs bypass pathsec and touch files outside allowed roots
GHSA-97qj-x29f-37w7highnltk@3.8NLTK: Entity-expansion DoS (billion laughs) via remaining raw ElementTree parses
GHSA-9r6g-266r-89x4highnltk@3.8NLTK Stanford wrapper classes execute untrusted JAR files without verification
GHSA-cc5r-64rf-75hghighnltk@3.8NLTK: Untrusted search path in prover9/mace4 config allows local code execution via relative binary_location
GHSA-cgvx-9447-vcchhighnltk@3.8ntlk unsafe deserialization vulnerability

By the numbers

Stars67.8K
Forks8,863
Contributors30
Commits437
Open issues33
Open pull requests55
Releases2
Latest releasev2
LicenceApache-2.0
Main languagePython
Project age2 years
Last pushNov 18, 2026
Tracked files49
Lines of code40.4K
Checkout size13 MB

Lines by language: JSON 31.5K, Python 4,717, Markdown 4,130.

Questions

Is MiniMind free?

Yes. The code is free under the Apache 2.0 licence, and the models and datasets are free downloads, though some datasets carry their own licences, including non-commercial ones. The only cost is GPU time if you train: the README estimates about 3 yuan, under half a US dollar, on a rented RTX 3090 for the quick version.

Can MiniMind replace ChatGPT?

No. At 64 million parameters it is a teaching model, about 1/2,700 the size of GPT-3, and its answers are far weaker than a big assistant's. Its value is that you can see, change and rerun every step that made it.

What hardware do I need?

Chatting with the finished model runs on an ordinary CPU. For training, an NVIDIA GPU with CUDA is the practical choice: the README's times are for one RTX 3090 with 24 GB, which you can rent by the hour. Training also runs on a CPU or an Apple Silicon Mac, much more slowly.

Does MiniMind send my data anywhere?

No. Training and chat run on your machine. It downloads models and data from ModelScope or Hugging Face when you ask, and sends training charts to SwanLab only if you add --use_wandb.

Do I need to know how to code?

Yes. You run Python scripts from a terminal, and you get the most from it if you can read PyTorch code. To just try the model, the README links an online demo on ModelScope.


This post is part of GitHub Tools, where every repository is cloned and scanned before it is written up. The scan is a snapshot of one commit on one day; the repository has moved on since, so check it before you install.