Last year, a consultant who built retrieval systems for more than ten regulated companies wrote up what actually matters once you pass 20,000 documents. Pharma, banks, law firms. The thread collected 969 upvotes on r/AI_Agents and earned them. I read all 28 pages so you don't have to.
Swap "pharma" for "construction" and almost every lesson lands. Here are the four that hold up on our projects.
The distance between a demo and a system a project team trusts is document scoring, structure-aware chunking, boring metadata, and honest failure handling.
01
Score your documents before you touch them
His pharma client had typewritten papers from 1995, scanned, sitting next to 500-page clinical reports with embedded tables. Sound familiar? Our projects mix stamped scans of old drawings, phone photos of site paperwork, and clean BIM exports. Run one pipeline over all of it and you get confident nonsense.
His fix was unglamorous: score extraction quality first, then route. Clean PDFs get full processing. Decent ones get cleanup. Garbage gets a flag for human review. He says this one change did more than any model upgrade. That matches what I see on real project document systems.
02
Chunk along the document's structure, or lose its meaning
Every tutorial says to cut text into 512-token blocks. A CCTP disagrees. It has lots, articles, tables, annexes, and a question about article 4.2 deserves the text of article 4.2, whole. He built hierarchical chunking (document, section, paragraph, sentence) and lets the question pick the level.
Our documents have more structure than pharma papers, which makes this easier for us, if we bother to use it.
03
Metadata beats the embedding model
He spent 40% of his time on metadata schemas and calls it his highest-return work. For pharma that meant document type, drug class, patient group. For us it means document type (CCTP, plan, PV, DOE), lot, phase, discipline, revision.
Two warnings worth repeating. First, an LLM extracting your metadata is inconsistent; simple keyword rules on 100 to 200 core domain terms work better. Second, the domain experts you already have are happy to build that term list. In our world, that expert is a works supervisor with twenty years of site time.
04
Semantic search fails more than anyone admits
He measures 15 to 20% failure in specialized domains, far above the 5% people assume. The failure modes read like a week on any project: acronym collisions ("EP" is rainwater on one sheet and street lighting on the next), exact-value questions ("the value in table 3" needs a rule, not a similarity), and reference chains (a spec points to a plan, the plan points to a standard; similarity sees none of it).
His answer: a relationship layer between documents, built at ingestion.
What I take from this
None of these four things shows up in a demo. All of them decide adoption. MIT reported that 95% of enterprise GenAI pilots fail; threads like this one show where the missing work lives: inside the document system nobody wants to open.
Next step on my side: I'm running these four lessons against a real CCTP set, and I'll publish what breaks.
Source
"Building RAG systems at enterprise scale (20K+ docs): lessons from 10+ enterprise implementations" by u/Low_Acanthisitta7686 on r/AI_Agents, September 2025. redd.it/1nbrm95