GPU inference
2 projects tagged GPU inference, in the order of our ranking. Some replace a paid product; others are free tools people use alongside one.
| Compare | Project | Runs on my machine | Allowed at work | Paid services | Hardware | Setup | Stars |
|---|---|---|---|---|---|---|---|
| vLLMOpen-source engine for serving LLMs at high throughput on your own GPUs, with an OpenAI-compatible API server; Apache-2.0. | Yes | YesApache-2.0 | None required | No graphics card needed | Medium | 96k+537 this week | |
| FreeTokenApache-2.0 engine and desktop app for running very large open models on your own NVIDIA GPU, with OpenAI and Anthropic-compatible APIs. | Yes | YesApache-2.0 | None required | GPU, about 8 GB | Medium | 16k |
Tick up to three to compare. “Allowed at work” includes projects with conditions, marked amber. Unknown is never counted as a yes. Stars show attention, not quality.
Know a project we’re missing? Suggest it.