Build · Beginner · An afternoon
Build a private chat-with-your-documents app with open-source software
Ask questions of your own PDFs, policies, manuals and letters, with nothing leaving your machine.
Instead of paying for ChatGPT Plus, Dropbox or Box.
What you'll build
A private assistant that has read your paperwork: drop in insurance policies, manuals, contracts, warranty cards or a textbook, and ask questions in plain words. "Does my policy cover water damage?" "What's the warranty on the dishwasher?" Answers point back to the page they came from. Unlike uploading files to a chat service, your documents stay on your own machine, which matters for medical bills, contracts and anything personal.
Blueprint
Files live in Nextcloud; AnythingLLM indexes them and answers questions with a local Ollama model.
- You PDFs, scans and letters Nextcloud Server
- Nextcloud Server The documents to index AnythingLLM
- AnythingLLM Your question plus the most relevant passages Ollama
- Ollama An answer that cites the page You
The repos
- 01 · Index and chat AnythingLLM
All-in-one desktop or Docker app to chat with your documents using local or cloud LLMs, with agents and a built-in vector database.
- Setup
- Easy; desktop installer, or Docker for multi-user source
- Hardware
- Recommended 16GB RAM, 8-core CPU; GPU recommended on Windows; storage varies by model source
- Paid extras
- None required; local LLM, embedder and vector DB by default; cloud optional
- 02 · Run the model Ollama
Local model runner that downloads and serves open LLMs (Gemma, Qwen, DeepSeek, gpt-oss and more) with a CLI and local API.
- Setup
- Easy; one-line install script or installer, then ollama run source
- Hardware
- Apple M-series or x86 (CPU only) on Mac; 4GB for Windows binary; RAM not stated source
- Paid extras
- None for local use; Ollama Cloud plans optional (Pro $20/month)
- 03 · Store the files Nextcloud Server
Self-hosted file sync and share server with desktop and mobile clients, share links, file versions and an app store for calendar, contacts and more.
- Setup
- Medium; All-in-One Docker image, appliances, or manual PHP/web server/database install source
- Hardware
- At least 128 MB RAM per PHP process, 512 MB recommended source
- Paid extras
- None required; Nextcloud Enterprise (paid support) and hosted providers are optional
Step by step
Install commands change, so each step links to the repo's own docs for the exact commands.
-
Install AnythingLLM and Ollama
The desktop app is the quickest start. Point it at Ollama as the language model so everything runs locally. AnythingLLM recommends 16 GB of RAM.
Uses AnythingLLM · its docs
-
Gather your documents
Keep the source files in one Nextcloud folder so they're backed up and reachable from your phone. Sync that folder to the machine running AnythingLLM.
Uses Nextcloud Server · its docs
-
Make a workspace per topic
Create separate workspaces such as Home, Insurance and Car. Answers are better when the model only searches the documents that matter.
-
Ask, and check the source
Ask questions and open the cited passage each time. For anything with money or deadlines, read the original.
Watch out
Answers are only as good as the model you run; check anything that matters, such as an insurance appeal, against the source.
Take it further
- Paste in an insurance denial letter and your policy, and ask what clauses support an appeal.
- Load a course textbook and ask it to quiz you on chapters you got wrong.
- Keep a warranty workspace and ask which appliances are still covered.
Questions
Does anything get uploaded?
Not with Ollama as the model and the default local settings. If you switch to a cloud model, your questions and passages go to that provider.
Can it read scanned paper?
Text in scanned PDFs needs to be readable text. Run old scans through OCR first if answers come back empty.
Can my family use it?
Yes. Run AnythingLLM with Docker for several users and give each their own workspaces.
Every repo here is open source, still maintained and checked against its own documentation. How we check →