Post-generation verification with corpus-curated retrieval for rheumatology AI: what works, what does not, and what we cannot claim

We describe a clinical AI verification system and report its limitations honestly. A candidate rheumatology response is scored on four dimensions (accuracy, safety, therapeutics, stewardship) by GPT-4o as evaluator — the same model family as the generator, a known limitation (Huang et al. ICLR 2024 showed LLMs often cannot self-correct factual errors without external grounding). Retrieved passages from a corpus of 81,502 articles partially mitigate this by providing external evidence. PCA on corpus embeddings concentrates domain-specific variance into fewer dimensions before quantisation; this preserves clinical distinctions that random rotation destroys (95% vs 87% recall at 10). This is NOT a claim that reducing bit precision improves retrieval — the improvement comes from better preservation of domain-relevant signal. Evaluation: 125 scenarios scored by LLM evaluator (not human rheumatologists). Combined system 8.90 vs 8.18 unaugmented. Naive retrieval with general embeddings scored 7.92 (worse than baseline). Hallucination assessed by manual review of 40 responses: 12-15% unaugmented vs under 2% verified — small sample, no inter-rater reliability. All benchmarks are internal. No independent validation exists. Statistical power is limited.