FreeToken
Apache-2.0 engine and desktop app for running very large open models on your own NVIDIA GPU, with OpenAI and Anthropic-compatible APIs.
FreeToken (~16k stars, nightly builds only so far) serves large Mixture-of-Experts models on personal machines by sharing the work between GPU, CPU and memory. A desktop app for Windows and Linux sets it up; on Linux there is also a Python package. It offers OpenAI and Anthropic-compatible APIs for coding agents. Needs an NVIDIA GPU; AMD support is experimental. Its authors' paper reports an 8GB laptop GPU running a 35B model and a gaming desktop a 284B one. Those claims were not tested here. Apache-2.0 and free; each model keeps its own licence.
The facts
| Question | Answer | Source | Checked |
|---|---|---|---|
| Runs on my machine? | Yes. Yes; serves models from your own machine, by default on 127.0.0.1 | github.com | Checked yesterday |
| Code licence | Open source. Apache-2.0 | github.com | Checked yesterday |
| Commercial use | Yes. Engine is Apache-2.0; commercial use of a model depends on that model's licence | github.com | Checked yesterday |
| Paid services needed? | None required; runs open models on your own hardware | flashml.ai | Checked yesterday |
| Hardware | NVIDIA RTX 30, 40 or 50 series; the paper's smallest machine is an 8GB laptop GPU; larger models need more; AMD is experimental | arxiv.org | Checked yesterday |
| Operating systems | Desktop app for Windows and Linux; the Python package is documented for Linux x86_64; macOS not mentioned | github.com | Checked yesterday |
| Setup effort | Medium; the desktop app sets up the engine, but you need a supported GPU and large model downloads | github.com | Checked yesterday |
Each fact has its own date. Unknown means nobody has published it. We read their documentation; we didn’t install it. Running it yourself costs your hardware, electricity and setup time. Report a correction.
No licence changes since we started watching. Licence changes: follow FreeToken and we email you if its terms change.