Tquestion-parsing brick of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks: parsing, question parsing, retrieval, and generation.
Question parsing is the second brick. This is the first of its three parts:
- why it exists, what it produces, and the split that motivates the rest.
- The next part covers what the parser extracts from a user string (Article 6 B, extraction) and
- how the parsed row is dispatched to retrieval and generation (Article 6 C, dispatch).
Article 1 (minimal RAG) showed the basic pattern: take the user’s question, send it to an LLM, and get an answer back.
The series pushed it a step further with prompt engineering, asking the LLM to return a JSON object with the answer plus the source passages it used. With a well-parsed document and structured output, finding where the answer comes from is no longer the hard part.
That’s the starting point. What’s left is the question itself, and that’s where most of the work in a real RAG pipeline ends up living. Three levels of “working the question” show up in enterprise projects, and they tend to arrive in this order.
1. Natural questions from users. A user types whatever comes to mind: “What’s the cap on liability?”, “Tell me about the exclusions”, “How does this compare to last year’s policy?” The system has to make sense of a string it didn’t author, in vocabulary that isn’t necessarily the document’s. This is the case most demos show, and the one with the most variance on the input side.
2. Templates the developer writes ahead of time. Pretty quickly, the team notices that the same questions come back over and over. For every contract, someone wants to know who the client is, who the insurer is, the annual premium, the guarantees, the exclusions, the renewal date. Rather than wait for users to type those one by one, the developer writes them once, jointly with the business team that owns the documents, and runs them across the whole corpus. The output is a JSON record per document. In practice this is what most companies build first; free-form chat tends to come later, on top of this base.
3. Helping users formulate. A pattern that shows up once a system is in front of real users: business teams know what they’re after, but have trouble phrasing it tightly. “What’s the cap?” leaves a lot open. The system can step in and ask back: “do you mean liability, indemnification, deductible, or premium?” Same machinery as the other two levels (a typed object with named fields), but now the empty fields drive a small dialogue with the user.
The three levels share the same machinery: turn the question into a structured object, then route the parts to retrieval and to generation. What changes is who fills the structure (the user, the developer, or both in dialogue) and what the system does when a field is empty (proceed with defaults, fail loudly, or ask back).
A user types one string into the chat box. “What is the maximum coverage amount? Don’t confuse it with the deductible, they’re often listed together.”
There’s a lot in that string. A topic (maximum coverage amount). An expected shape (an amount, a number). A negative cue (don’t confuse with deductible). A hint about how the document presents the answer (often listed together). If you hand the whole string to retrieval, none of those parts land where they should: the embedding pulls deductible-bearing lines closer, the format hint never reaches generation, the disambiguation gets either embedded with the rest or stripped at preprocessing. The result is a confident wrong answer (the deductible’s number, not the coverage’s), and a retriever that no observability dashboard can debug.
The fix is to parse the question first. Turn the noisy user string into a typed, structured brief that downstream bricks can act on. Then split the brief in two: the retrieval brick reads what it can act on (topic, rewrites, scope), the generation brick reads what it can act on (the original wording, the format constraint, the disambiguation). Neither brick gets confused by the other’s signal. Both stay focused on what they do best.
1. Question parsing mirrors document parsing
1.1 Same approach: a relational set of tables
Document parsing produces a relational set: line_df with one row per text line, page_df with one row per page, toc_df with one row per TOC entry, plus a few satellites.
Question parsing has the same goal: turn the unstructured input into structured form before the next steps run on it. The artifact is the same kind of thing: a relational set.
The shape diverges in one obvious way. A document fills line_df with hundreds or thousands of rows. A question is a single string, so it fills one row in a question_df table. The columns of that row are what the parser computes: the spell-corrected text, the extracted keywords, the type of answer expected, the pages or sections the user mentioned, the decomposition pattern, the activations the dispatcher should turn on. Adding a parsing capability is adding a column.
The other half of the relational picture is satellite tables the question row links into. They’re as open-ended as the columns themselves: a project starts with the ones it needs, and adds others as new cases push for them.
The most common one is the project’s expert keyword dictionary, split in two: concepts_df (one row per concept, with its definition) and concept_keywords_df (one row per (concept, language, keyword)). Together they hold the domain-specific synonyms: premium → prime, cotisation, tarif annuel; non-compete → restrictive covenant; side effects → adverse events. Another we’ll meet later is answer_types_df, which registers the kinds of answers questions can expect (amount, date, iban, …).
Real projects usually grow more:
- a
regulations_dfmapping legal codes to their texts for a legal RAG - an
entity_alias_dffor company name variants in a corporate corpus - a
unit_conversions_dfin a scientific one
The question’s columns link to these satellites the same way line_df linked to image_df in document parsing.
This relational framing matters in practice. Once questions are rows in a table, you can SQL them: “how many questions of type amount did users ask last month?”, “which questions triggered a clarification request?”, “which expert-dictionary entries got hit most often?”. The question history becomes ops data, not just a log file. A follow-up storage chapter develops the layout that makes question_df a first-class table.
A note on terminology. The usual names for this are “query understanding” and “question understanding.” Both are vague: they suggest the system understands the question, which doesn’t say much. Question parsing names what happens: take a string, return an enriched relational row.
1.2 Where this fits in pdf_qa
Article 1 (minimal RAG) introduced the top-level call as a single function: result = pdf_qa(contract_pdf, question="What is the maximum coverage amount?"). The user passes a PDF path and a question, gets back a structured JSON answer. The name follows the pdf is the format, qa the intent. Volume 2 opens both axes (excel_qa, pdf_translate, …) with a dispatcher doc_qa; Volume 1 calls the handler directly.
Inside, pdf_qa wires the four bricks in sequence: parse_pdf → parse_question(question, doc_profile=...) → retrieve → generate, then wraps the answer with an _meta audit block. The extraction companion (Article 6_b) details which columns the parser fills from the user string; the dispatch companion (Article 6_c) details how each column is routed to the brick that consumes it.
The rest of this article covers the thesis: why splitting the parsed row into two consumer briefs is the right move, and the one specific case that proves it (negative cues like “not the deductible”).
2. Two consumer briefs from one parsed row
The question row in question_df carries everything the pipeline could possibly need, but retrieval and generation don’t need the same subset. The parser emits two derived views of the row, each shaped for the stage that consumes it: RetrievalQuery and GenerationBrief. Why split them, and not just hand the full row to both?
2.1 The split between retrieval and generation
Because the two consumer bricks have very different strengths:
- Retrieval is similarity matching: Good at finding what’s close to the query. Bad at rejecting precisely. It cannot tell you what’s related to but not equal to the query.
- Generation is reading and reasoning: Good at distinguishing, excluding, contrasting. It can hold both concepts in mind, compare them, and select one. It’s the only stage in the pipeline that can do this.
Each derived view contains only the columns its consumer can act on. The retrieval brief gets the topic, rewrites in document vocabulary, anchor keywords (codes, IDs), and the scope filters that pre-filter the candidate space. The generation brief gets the original question (so the user’s intent is preserved), the format constraint, the disambiguation cues, and any distractors the LLM should not confuse with the real answer.
The most common mistake is to send everything to both. The retrieval side gets confused by “don’t confuse with deductible” (it has no way to act on a negation); the generation side gets confused by rewrites (it should answer in the user’s terms, not the document’s). The two derived views keep each stage focused on what it can do.
2.2 Why exclusions belong to generation, not retrieval
A natural reflex, when a user says “don’t include X”, is to filter X out at retrieval. Don’t. It’s almost always the wrong move.
Take a concrete example. The user asks:
“What is the limit per claim, not the deductible, in this contract?”
The naive approach is to remove from retrieval any passage that mentions “deductible”. Three problems, one for each level you might try to exclude at:
Problem 1: line-level exclusion: Suppose you exclude any line containing “deductible”. Then you lose lines like:
“The limit per claim is €1,500,000, with a deductible of €1,000.”
That line is the answer. The exclusion just deleted it. Limit and deductible are often stated next to each other precisely because they’re related: that’s the whole reason the user warned you about the confusion.
Problem 2: page-level exclusion: Even worse. The page that contains the limit also contains the deductible (in the same table, in the same section). Excluding the page throws away the answer entirely.
Problem 3: section-level exclusion: Worst of all. The section “Limits and Deductibles” is, by name, exactly the section that contains the answer. Excluding it removes everything relevant.
Beyond granularity, there’s a bigger issue. Embeddings don’t do exclusion. Adding “NOT deductible” to the query string doesn’t work. Embeddings ignore negation, a failure mode walked through with measurements in Article 2 (embeddings’ failure modes). The vector for “limit per claim NOT deductible” is almost identical to the one for “limit per claim deductible”. You haven’t excluded anything. You’ve just added a word. BM25 with negative queries is fragile too. You can write query = "limit per claim" AND NOT "deductible" in some retrieval engines. But this excludes any passage where both words appear, including the exact passages that contain the answer alongside its contrast.
The right answer is simple: retrieve broadly, exclude at generation. Retrieval brings back any passage where “limit” or “deductible” or “claim” is mentioned. The generation step receives all of them, plus the explicit instruction “the user is asking about the limit, not the deductible.” The LLM then reads the passages, identifies the limit, reads the deductible, and reports only what was asked.
The pattern repeats: detect the disambiguation cue at the parser, route it to the generation brief, let the LLM apply it.
Once this split clicks, the same logic applies to almost any negative instruction: “…not from the previous version” → retrieve all versions, filter at generation. “…except the optional clauses” → retrieve all, filter at generation. “…without the marketing language” → retrieve all, summarize cleanly at generation.
Retrieval should be broad. Generation should be selective.
3. What comes next
The question is parsed. The two consumer briefs are ready. Two follow-up articles close the brick:
- Article 6 B (extraction) walks what the parser extracts from a user string. Keywords (with several sources combined), the expected answer shape and type, scope hints, decomposition for compound questions, and the clarification field for vague inputs. Each becomes a column on
question_df. - Article 6 C (dispatch) walks how the parsed row is routed. The dispatch decisions (chunk strategy, model, answer context) the parser makes on top of what the user said, using the document’s profile. The activation flags that adapt the pipeline to the document. And the audit
_metablock that records every decision for replay.
Each of those two articles stands on this one’s thesis: parse first, route by brief, never let a generation-only signal pollute retrieval.
4. Sources and further reading
The two-brief shape this article argues for is a refinement of “function-calling-style” question parsing that most production RAG systems converge on once free-form chat hits its first hundred users. The right cross-reading is the original embedding-failure paper (where retrieval-side negation breaks down empirically) and the production RAG playbooks that converged independently on the same answer: broad retrieval, strict generation.
Same direction as the article:
Different angle:
- Most RAG-in-a-day tutorials treat the question as a black box passed verbatim to the embedder. The thesis here is the opposite: the question carries structure (topic, scope, shape, disambiguation) that the retrieval and generation stages need to read separately. Adopting that view is what makes the brick exist.
Earlier in the series:
Part I:
- Baseline Enterprise RAG, from PDF to highlighted answer. The four-brick pipeline end to end: PDF in, highlighted answer out.
- Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. Where embedding similarity wins (synonyms, typos, paraphrase), where it predictably breaks (unknown terms, negation, term-vs-answer relevance), and how to use it anyway.
- Rerankers Aren’t Magic Either: When the Cross-Encoder Layer Is Worth the Cost. What a cross-encoder adds over bi-encoder embeddings, measured, and when it is worth the latency.
- RAG is not machine learning, and the ML toolkit solves the wrong problem. Why chunk-size sweeps and finetuning optimize the wrong thing; route by question type instead.
- From regex to vision models: which RAG technique fits which problem. Two axes, document complexity and question control, that pick the technique for each case.
- 10 common RAG mistakes we keep seeing in production. Ten production mistakes, organized brick by brick, with the fix for each.
Part II:
Leave a Reply