# FreeToken

> FreeToken (~16k stars, nightly builds only so far) serves large Mixture-of-Experts models on personal machines by sharing the work between GPU, CPU and memory. A desktop app for Windows and Linux sets it up; on Linux there is also a Python package. It offers OpenAI and Anthropic-compatible APIs for coding agents. Needs an NVIDIA GPU; AMD support is experimental. Its authors' paper reports an 8GB laptop GPU running a 35B model and a gaming desktop a 284B one. Those claims were not tested here. Apache-2.0 and free; each model keeps its own licence.

- Repository: https://github.com/FlashML-org/FreeToken
- Official docs: https://github.com/FlashML-org/FreeToken#readme
- Checked: 2026-11-02

## What it replaces
Nothing people pay for: a free tool, listed because people search for it.

## Facts, with sources

| Question | Status | Answer | Source | Checked |
|---|---|---|---|---|
| Runs on my machine? | Confirmed | Yes; serves models from your own machine, by default on 127.0.0.1 | https://github.com/FlashML-org/FreeToken/blob/main/docs/quickstart.md | 2026-11-02 |
| Code licence | Confirmed | Apache-2.0 | https://github.com/FlashML-org/FreeToken/blob/main/LICENSE | 2026-11-02 |
| Model weights licence | Confirmed | No bundled weights; each model you load keeps its own licence | https://github.com/FlashML-org/FreeToken/blob/main/docs/models.md | 2026-11-02 |
| Commercial use | Confirmed | Engine is Apache-2.0; commercial use of a model depends on that model's licence | https://github.com/FlashML-org/FreeToken/blob/main/LICENSE | 2026-11-02 |
| Paid services needed? | Confirmed | None required; runs open models on your own hardware | https://www.flashml.ai/ | 2026-11-02 |
| Hardware | Partial | NVIDIA RTX 30, 40 or 50 series; the paper's smallest machine is an 8GB laptop GPU; larger models need more; AMD is experimental | https://arxiv.org/abs/2608.16157 | 2026-11-02 |
| Operating systems | Partial | Desktop app for Windows and Linux; the Python package is documented for Linux x86\_64; macOS not mentioned | https://github.com/FlashML-org/FreeToken#desktop-app | 2026-11-02 |
| Setup effort | Confirmed | Medium; the desktop app sets up the engine, but you need a supported GPU and large model downloads | https://github.com/FlashML-org/FreeToken#desktop-app | 2026-11-02 |

GitHub: 16076 stars, 1586 forks, last push 2026-11-02, archived: no.

## Alternatives to FreeToken
- [Tauri](https://repoquarry.dev/projects/tauri.md): Framework for small desktop and mobile apps with a web front end and a Rust back end, using the system's own web view instead of bundling Chromium.
- [FastAPI](https://repoquarry.dev/projects/fastapi.md): Python web framework for building APIs from type hints, with automatic validation and interactive OpenAPI docs; MIT licensed.
- [vLLM](https://repoquarry.dev/projects/vllm.md): Open-source engine for serving LLMs at high throughput on your own GPUs, with an OpenAI-compatible API server; Apache-2.0.
- [Ghostty](https://repoquarry.dev/projects/ghostty.md): Fast, native, GPU-accelerated terminal app for macOS and Linux, plus libghostty for building terminals into other apps; MIT.
- [Herdr](https://repoquarry.dev/projects/herdr.md): Apache-2.0 terminal workspace manager for coding agents. Runs Claude Code, Codex and others side by side and shows which one needs you.
- [FreeLLMAPI](https://repoquarry.dev/projects/freellmapi.md): Self-hosted router that pools free-tier LLM provider keys behind one OpenAI-compatible API, for developers and coding agents.
---
Source: RepoQuarry (https://repoquarry.dev/projects/freetoken). Data licence: CC BY 4.0. Quotes are from project documentation and belong to their authors.