RAG vs Fine-Tuning Matrix: 17 Evidence-Bound Rules

Q: Does RAG guarantee correct citations?

No. Retrieval can preserve document and chunk provenance, but the system must separately test whether each generated claim is supported by the cited passage.

Q: When should a team combine RAG and fine-tuning?

A hybrid is a candidate when the workload independently needs changing or traceable knowledge and stable specialized behavior, and the team can operate both lifecycles.

Direct answer

Choose by the gap you must close

Use retrieval for changing or traceable knowledge, parameter adaptation for stable specialized behavior, and a hybrid only when those needs are independent. Start with a measured baseline when the problem, examples, corpus, or evaluation set is not yet ready.

RAG candidate

Best fit when answers depend on maintained external facts or source passages and the team can own ingestion, indexes, access controls, retrieval evaluation, and rollback.

Fine-tuning candidate

Best fit when the primary gap is repeatable task behavior, format, classification, extraction, or instruction following, backed by representative examples and regression tests.

Hybrid candidate

Best fit when changing evidence and specialized behavior are both required, and retrieval and model releases can be tested and rolled back separately.

Baseline or evidence first

Use prompting, long context, a deterministic workflow, or a data/evaluation task first when added architecture has no proven job or a required dependency is missing.

Decision flow from knowledge and behavior requirements to RAG, fine-tuning, hybrid, or baseline-first candidates — The method separates knowledge requirements from behavior requirements, then checks data and operating readiness. Every candidate still needs a workload-specific evaluation.

Interactive classifier

Turn architecture assumptions into a decision record

Answer with evidence you can name. Choose Unknown when an owner, artifact, or measurement does not exist. The engine evaluates hard blockers before candidates and attaches advisory tests without changing the result.

Architecture matrix

What each option changes, owns, and can fail

A comparison is useful only when it names the data asset, update path, runtime path, evidence boundary, and operating surface. "Accuracy" is not a static property of an architecture label.

Qualitative comparison for selecting the next experiment. Every row requires workload-specific validation.
Decision factor	Prompt / long context	RAG	Fine-tuning	Hybrid
Primary job	Establish the simplest measurable behavior and context baseline.	Supply external knowledge at request time.	Adapt model behavior or a stable task contract.	Separate changing knowledge from trained behavior.
Knowledge location	Request context supplied directly.	Permissioned corpus plus retrieval index.	Model parameters or an adapter; not a dependable live fact store.	Corpus/index for facts and model version for behavior.
Update path	Change prompt, tools, or supplied context.	Ingest, validate, index, and invalidate changed content.	Revise examples, train, evaluate, and release a model version.	Two separate update and rollback paths.
Required data asset	Representative tasks and accepted outputs.	Authoritative corpus with identity, permissions, and update ownership.	Representative permitted examples plus held-out regressions.	Both assets, aligned to the same task contract.
Provenance potential	Only for context the application explicitly preserves.	Document and chunk IDs can travel with retrieved passages; citation support still needs testing.	Training examples do not provide claim-level provenance at inference time.	Retrieval can provide evidence while tuning controls how it is used.
Runtime path	Request, context, model, response.	Request, authorization, search, rerank, context assembly, model, response.	Request, selected model version, response.	Retrieval path plus tuned-model path.
Main failure class	Prompt fragility, missing context, or context overload.	Missing, stale, unauthorized, distracting, or ignored evidence; citation mismatch.	Overfit, regressions, stale memorized facts, weak examples, or version drift.	Both failure classes plus interaction and attribution errors.
Cost, latency, privacy	Measure on the intended provider, corpus, model, traffic shape, quality target, data flow, and operating process. No technique has one universal advantage.

Machine-readable version: decision-matrix.csv. Rule data: rules.json.

Disqualifiers and break conditions

A plausible label cannot repair a missing dependency

The engine tests these conditions before it selects a candidate. A blocked result is an architecture finding, not an invitation to lower the evidence bar.

Fresh facts, no governed corpus

Fine-tuning cannot substitute for an authoritative source. Identify the system of record, content owner, permissions, stable identifiers, update events, and deletion path first.

Behavior goal, no examples

Without representative success and failure examples, tuning has no reliable task contract. Establish a prompt baseline and collect permitted evidence before training.

No evaluation set

Without versioned tasks and acceptance criteria, a demo can look better while regressing important cases. The tool returns insufficient evidence.

Citations without faithfulness

A retrieved source ID proves that content was fetched, not that the generated sentence follows from it. Evaluate material claims against cited passages.

Privacy by architecture label

RAG context may cross a provider boundary; training data may do the same. Map storage, retrieval, training, retention, access, logging, and deletion for the exact services used.

Hybrid before isolated wins

Combining two unproven subsystems hides failure attribution. Test retrieval and behavior adaptation separately, then combine only when each closes a measured gap.

Retrieval design

RAG is a governed knowledge path, not a vector database checkbox

A production retrieval system must transform a user need into authorized evidence, preserve provenance, and expose failures at each stage.

Corpus and ingestion

Define authoritative sources, document identity, ownership, permissions, effective dates, deletion semantics, and an update trigger. Chunking should preserve enough local context to support retrieval and citation. An embedding refresh that fails silently is a freshness failure.

Lexical and semantic retrieval

Embeddings recover semantic similarity. Lexical retrieval such as BM25 can recover exact product codes, legal references, names, and rare phrases. When both signals matter, evaluate each path, fusion, and reranking against labeled queries rather than assuming one retriever is sufficient.

Reranking and context assembly

A reranker can reorder the initial candidate set. Context assembly then applies authorization, diversity, freshness, token budget, and deduplication rules. Record document and chunk IDs, index version, query transformation, scores, and filters for diagnosis.

Grounding and citations

The generator may ignore a relevant passage, combine incompatible passages, or cite text that does not support the claim. Measure retrieval coverage and claim-to-passage support separately from answer fluency. Design an abstention path for missing or conflicting evidence.

The scope of this instrument begins with Pharos Production's engineering comparison of RAG and fine-tuning, then narrows its quantitative claims through primary-source review and turns the qualitative choice into versioned rules.

Model adaptation

Fine-tuning needs a stable behavior contract and a release discipline

Training is justified when examples describe a repeatable target better than instructions alone, and when the team can detect both task gains and regressions.

Supervised fine-tuning

Supervised fine-tuning learns from input-output examples. Suitable targets include stable formatting, classification, extraction, instruction following, and domain-specific response behavior. Train/validation separation, data rights, contamination checks, class coverage, and failure examples belong in the release evidence.

Preference optimization

Preference methods learn from ranked or judged outputs rather than only one target answer. They still require a defined behavior objective, consistent labels, held-out evaluation, and regression checks. They are not a live knowledge synchronization mechanism.

PEFT and LoRA

Parameter-efficient fine-tuning updates a constrained parameter set. LoRA freezes pretrained weights and learns low-rank update matrices. This changes the adaptation mechanism, not the need for representative data, model-version provenance, provider support, evaluation, and rollback.

Release record

Bind the base model, provider or weights digest, tokenizer, dataset version, split, recipe, adapter or tuned artifact, evaluation suite, safety checks, approval, deployment target, and rollback artifact. Re-run regressions when any bound input changes.

Do not train changing facts as the primary store

Facts encoded in parameters are difficult to enumerate, attribute, edit, or remove one by one. If the requirement is current, permissioned, or inspectable knowledge, keep that knowledge in a governed source path and treat model behavior separately.

Hybrid architecture

Let retrieval own facts and tuning own behavior

A hybrid is not a compromise score. It is a separation of responsibilities that creates two independently versioned systems and one combined evaluation surface.

Hybrid architecture with separate retrieval-index and tuned-model build paths feeding one request-time response path — At build time, corpus/index versions and training/model versions remain separate. At request time, authorized retrieved evidence and the user request enter the selected model. Observability must attribute failures to the retrieval, generation, behavior, or control layer.

Use it when

The workload requires maintained external evidence and a stable specialized response contract; the team already has representative examples, an authoritative corpus, an evaluation set, and owners for both lifecycles.

Test it in stages

First test the prompt baseline. Then isolate retrieval. Then isolate adaptation. Combine them only after each closes a measured failure class. Include distractors and missing-evidence cases when testing whether the tuned model uses context correctly.

Avoid it when

One component has no independent job, operations are unowned, a simpler context window works, or the evaluation set cannot locate which layer failed. Complexity without attribution is operational debt.

Worked decision patterns

Three candidate architectures and one refusal

These are rule demonstrations, not reported client outcomes. Each starts with a requirement, names the evidence pattern, and ends with a first experiment.

RAG candidate

Changing support knowledge base

Situation: Answers must reflect maintained policies and let users inspect the passages used.
Why: The primary gap is a traceable knowledge path, not a new behavior contract.
First experiment: Compare the current prompt with retrieval. Score passage relevance, claim support, answer quality, authorization, and stale-index failures separately.

Fine-tuning candidate

Stable structured extraction

Situation: A stable input class must produce a consistent schema; permitted representative examples already exist.
Why: The target is repeatable task behavior rather than changing external facts.
First experiment: Compare prompt-only and tuned candidates on held-out schema validity, task errors, preserved capabilities, versioning, and rollback.

Hybrid candidate

Domain assistant with specialized behavior

Situation: The assistant needs a changing domain corpus and a stable response contract represented in examples.
Why: Knowledge and behavior gaps are independent, and both lifecycle capabilities exist.
First experiment: Test retrieval and adaptation separately. Combine only if each improves its assigned layer without unacceptable regressions.

Insufficient evidence

Stakeholder-selected architecture, no evaluation set

Situation: A team asks for RAG or tuning, but cannot name versioned tasks or acceptance criteria.
Why: Alternatives cannot be compared against the same workload contract.
First experiment: Freeze representative tasks, failure classes, baseline outputs, review policy, and acceptance rules before implementing either path.

Evaluation contract

Measure the layer that can falsify the choice

Run candidates on the same representative tasks. Store inputs, versions, retrieved evidence, responses, review outcomes, latency, cost, access decisions, errors, and rollback evidence. A single aggregate "quality" number cannot locate the failing layer.

Evaluation loop from representative tasks through architecture candidates, layer-specific checks, failure analysis, and a release or stop gate — The evaluation loop can release, continue, or stop. Failure analysis should change one assumption or component at a time and preserve the evidence record.

Retrieval layer

Query and filter correctness
Passage relevance and coverage
Exact-term and semantic failures
Freshness and deletion propagation
Authorization and tenant isolation

Generation and behavior

Claim support and citation faithfulness
Task and format correctness
Abstention on missing evidence
Distractor resistance
Regressions on preserved capabilities

Operating contract

End-to-end latency and cost
Provider and model availability
Data retention and access paths
Observability and incident diagnosis
Version rollback and recovery drills

No universal acceptance threshold is embedded here.

Set thresholds from business loss, review capacity, risk classification, user expectations, and the measured baseline. Record them before comparing candidates so the target does not move after results are known.

Evidence and machine-readable assets

Trace each recommendation to a rule, claim, and source

The public data separates observed source statements from bounded architecture rules. Sources were checked on 18 August 2026; vendor features and policies can change after that date.

Decision registries

Evidence registries

Implementation record

Ownership, review, and limits

What was reviewed and what was not

Publisher: Pharos Production. Contact: info@pharosproduction.com.
Method and data version: 2026-08-18.1. Published and source-checked on 18 August 2026.
Production disclosure: Research synthesis, rule drafting, implementation, and deterministic checks were AI-assisted. Claims are bounded to the visible ledger.
Review boundary: Independent human review is not claimed. The release gate validates source scope, rule traceability, tests, structured data, semantics, and public artifacts.
Not included: No universal cost, latency, accuracy, dataset-size, document-count, privacy, or model-performance benchmark; no legal or security certification.
Corrections: Send the page URL, claim or rule ID, contrary source, and proposed correction by email. Accepted changes receive a new version and changelog entry.

Boundary questions

Questions the matrix treats as architecture constraints

Can fine-tuning keep changing company facts current?

Fine-tuning changes model parameters. Changing facts still need a governed update and release path; a maintained retrieval source is usually the clearer knowledge boundary. A model retraining cycle may be appropriate for some stable domain adaptation, but it does not make individual facts easy to attribute, edit, or delete.

Does RAG guarantee correct citations or eliminate hallucinations?

No. Retrieval can preserve document and chunk provenance. The retriever can still miss relevant evidence, return stale or distracting text, or cross an access boundary. The model can ignore the context or attach an unsupported citation. Test retrieval and claim-to-passage support separately.

Is a larger context window a replacement for RAG?

It can be the right baseline when the permissioned working set fits comfortably, update and access rules remain manageable, and the workload does not need a separate search or provenance layer. Measure quality, latency, cost, context-selection errors, and operational simplicity against retrieval instead of deciding from context size alone.

When should a team combine RAG and fine-tuning?

Use a hybrid candidate when the workload independently needs maintained external evidence and stable specialized behavior, representative examples and an authoritative corpus exist, and the team can operate and roll back both lifecycles. Test the components separately first.

Does PEFT or LoRA remove the need for a fine-tuning evaluation?

No. PEFT changes how parameters are adapted. LoRA learns low-rank updates while freezing pretrained weights. Neither method establishes data quality, task fit, preserved capabilities, provider availability, model provenance, or rollback evidence.

Which option is more private, faster, or cheaper?

The label is insufficient. Compare the exact corpus, provider endpoints, model and index hosting, retention settings, token path, cache, traffic shape, quality target, evaluation labor, update cadence, monitoring, and incident process. A vendor policy can also change after this page's verification date.