01 · The problem
How do you make a PDF queryable without sending the whole document to the LLM?
Large documents create a simple engineering problem: the useful answer may be buried hundreds of pages away, while passing the full document into every prompt increases context size, latency and cost.
The project therefore treats the PDF as a retrieval problem first and a generation problem second. The system finds the most relevant pieces of the document, then gives only that evidence to the language model.
02 · System architecture
A retrieval pipeline before the LLM
The core design separates ingestion from question answering. Documents are processed once, indexed, and then reused across queries.
Document
Chunk
& Index
Retrieval
The LLM is deliberately kept at the end of the pipeline so retrieval quality determines what evidence reaches generation.
03 · Engineering decisions
Where the engineering work matters
Chunking
Split documents into retrievable units instead of treating an entire PDF as one context block. Chunk size and overlap directly influence recall and prompt size.
Embeddings
Represent chunks semantically so a question can retrieve conceptually related passages rather than relying only on exact keyword matches.
Vector retrieval
Use similarity search to narrow a large document collection down to a small set of candidate passages before generation.
FastAPI boundary
Keep ingestion and query operations behind clean API boundaries so the retrieval layer can evolve independently from the client.
04 · Request lifecycle
What happens when a user asks a question?
- The API receives the user's question.
- The question is converted into the same semantic representation used for indexed document chunks.
- The retrieval layer searches for the most relevant chunks.
- The selected context is assembled into a constrained generation prompt.
- The LLM produces an answer grounded in the retrieved document evidence.
05 · Technology
Stack used for the implementation
It demonstrates the practical boundary between an LLM demo and an AI application: document ingestion, retrieval quality, API design, prompt construction and cost-aware context selection all have to work together.
06 · Continue reading
Related engineering work
Building an AI product?
I can help with the backend, RAG pipeline, LLM integration and production architecture.