The problem with confident wrong answers
A language model will always give you an answer. In a consumer chatbot that is a feature; in an enterprise workflow it is a liability. When an agent advises on a regulated procedure, a contract clause, or a patient-facing recommendation, a fluent-but-fabricated response is worse than no response at all.
Our position is simple: an enterprise agent should answer only from grounded, retrievable sources, cite exactly where each claim came from, and say "I don't know" when the knowledge base does not support an answer. We call this a hard data wall.
The retrieval pipeline
Every answer flows through the same four stages, and each stage is independently testable:
- Embedding — the query and source documents are projected into a shared vector space.
- Vector store — candidate passages are retrieved by semantic similarity.
- Re-ranking — candidates are re-scored against the query so the most relevant context rises to the top.
- Grounded generation — the model composes an answer constrained to the retrieved passages, with inline citations.
Hard data walls in practice
The generation step is prompted and constrained so the model cannot range beyond what retrieval returned. If the retrieved context does not contain the answer, the agent is required to say so rather than fill the gap from its parametric memory.
This is the single most important design decision for trust. It turns the knowledge base — not the model's training data — into the source of truth, which means the answer is auditable and improves the moment you update your documents.
Measuring what matters
You cannot ship trust you cannot measure. We run an offline, RAGAS-style evaluation harness against a curated gold set of questions before anything reaches production, tracking the metrics that actually predict real-world reliability:
- Context relevance — did retrieval surface the right passages?
- Faithfulness — is every claim in the answer supported by the retrieved context? Target: ≥95%.
- Hallucination rate — how often does the answer assert something unsupported? Target: under 5%.
- Latency — is the end-to-end response fast enough to feel instant? Target: ≤3 seconds.
Offline-first, with optional accelerators
Sensitive deployments cannot always call out to a third-party API. The same pipeline runs fully offline using a hashing embedder and an in-process vector store — no API keys, no network egress — so data never leaves your infrastructure.
When you do have GPUs and connectivity, the same code transparently upgrades to stronger sentence-transformer embeddings and a FAISS index for scale. The architecture does not change; only the components behind the interface do.
Why this matters for the enterprise
Grounded, cited, evaluated retrieval is what separates a demo from a system you can put in front of auditors, clinicians, or counsel. It is the foundation every FEME agent is built on — and the reason our agents can be trusted with decisions that carry real consequences.