Series 4 — Retrieval-Augmented Generation

A private question-answering system over your own PDFs, Office documents and notes — with particular attention to Polish-language material, which most default settings handle badly.

This is the workload that justifies the machine to people who are unmoved by generated video. Documents you would never upload to a third party — contracts, medical records, financial statements, years of personal notes — become searchable and answerable, and nothing leaves the box.

It is also the one workload here that genuinely needs several models resident at once: a generation model, an embedding model and a reranker. The good news is that the last two are small. The arithmetic works comfortably, and the series is explicit about it.

1. What a Private RAG Should Look Like Here lead The component map and the honest case for and against: when retrieval beats simply pasting a long document into a model, measured performance on this exact hardware, the multilingual retrieval problem, and the choice between assembling a stack and deploying a finished one.
2. Ingestion: Docling, Chunking, and Documents That Fight Back build Turning PDFs, Office files, scans and a folder of markdown notes into retrievable text. Layout-aware parsing, tables that survive, OCR for Polish scans, heading-aware chunking, and the metadata that makes citation possible.
3. Retrieval: Hybrid Search, Reranking, and Why Polish Breaks Defaults build bge-m3 doing dense, sparse and multi-vector retrieval at once, a cross-encoder reranker as the quality step that matters most, Qdrant as the store, embeddings served from vLLM, and the morphology problem that makes Polish keyword search behave differently from English.
4. The Answering Layer: API, UI, Citations, and Evaluation build An OpenAI-compatible endpoint that answers with sources, a browser interface, orchestration under the one-model-at-a-time discipline, and a small evaluation harness so that changes to the pipeline can be shown to be improvements rather than assumed to be.