ENGINEERING / MONITORING

PRODUCTION SYSTEMS / 08

Production AI
Drift Planner

PLANNING TOOL
v1.0.0 ·

Decide what to watch, when to investigate and what can safely roll back.

Use this worksheet to turn the production AI scope of Pharos Production into a monitoring plan your team can own. It runs in your browser. Your inputs stay in this tab.

Build your monitoring plan

01 / CONFIGURE
Application
Observation policy

Illustrative values. Apply the sample floor to each eligible denominator and cohort. Events bypass it.

Response ownership
YOUR PLANDRAFT · CALIBRATION REQUIRED

RAG application

Set thresholds below, then export the current configuration.

Signals included
10
Requests / window
1,000
Evaluations / window
100
REFERENCE 14 daysCOMPARE 24 hoursINVESTIGATE 2 windows

These are traffic estimates, not observed eligible samples. A restricted denominator, incomplete telemetry or delayed labels can make the usable count lower. Grounding counts claims. Its request estimate is only a sampling budget.

Exports contain definitions, thresholds, source IDs and response rules.

No live telemetry connection. Alerts and rollback actions remain with your team. Reloading clears edits.

Signals and response rules

02 / TUNE

Thresholds below are starting examples, not recommended production limits. Every definition states what belongs in the denominator. Event checks have their own response timing.

Unreviewed releases and unauthorized effects use a fixed zero-tolerance trigger. Other thresholds are editable.

Request telemetry

Input distribution shift

Total variation distance = 0.5 × sum of absolute differences in category proportions between current and reference prompts. Freeze the same intent/language taxonomy and include an unknown category. Denominator: prompts with a recorded category.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
A changed request mix can invalidate an old evaluation set without proving a quality regression.
Investigate
Inspect affected intent and language slices. Compare downstream outcomes and expand the holdout set.
Rollback condition
Do not roll back for input shift alone. Consider a release rollback only if a matched-cohort evaluation links degradation to that release.
Evidence
S2 · S3. Definitions and thresholds are this planner's design choices.
Evaluation

Task success rate

100 × evaluated completed tasks that pass the fixed task rubric / all eligible completed tasks evaluated. Record unscored and pending labels separately.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
Business outcomes can change while input distributions remain stable.
Investigate
Re-score a matched sample, examine labels and group failures by release, task and language.
Rollback condition
Use the last compatible release only after a repeated quality breach is attributable to a change and the candidate passes the same eval set.
Evidence
S2 · S6. Definitions and thresholds are this planner's design choices.
Evaluation

Benign-request refusal rate

100 × refused requests judged answerable and allowed / evaluated requests judged answerable and allowed. Exclude unsafe prompts from this denominator and monitor unsafe acceptance separately.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
A rise may indicate a policy or prompt regression. An appropriate safety refusal is not a failure.
Investigate
Review benign refusals with a stable labeling rubric before changing guardrails.
Rollback condition
Revert an implicated policy or prompt change only after both benign-helpfulness and unsafe-request tests pass.
Evidence
S2 · S6. Definitions and thresholds are this planner's design choices.
Request telemetry

Schema conformance

100 × outputs passing the versioned required schema / requests that require that schema. Include missing, truncated and unparsable outputs as failures. Track endpoint errors separately.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
Successful HTTP responses can still break downstream consumers.
Investigate
Compare first-attempt and final outputs separately. Inspect schema and model versions.
Rollback condition
Revert the implicated model, prompt or schema change if the prior compatible bundle restores validation without violating current requirements.
Evidence
S1 · S6. Definitions and thresholds are this planner's design choices.
Request telemetry

End-to-end p95 latency

95th percentile of end-to-end completed request durations, measured from application receipt to terminal response. Record timeouts/cancellations separately. Do not average per-host percentiles.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
A quality-preserving release can still exceed the user-facing latency budget.
Investigate
Inspect queue time, retrieval, retries and model spans. Compare like-for-like workload slices.
Rollback condition
If a canary release causes an SLO breach, route to a compatible known-good path. Check quality and capacity before expanding traffic.
Evidence
S5. Definitions and thresholds are this planner's design choices.
Evaluation

Grounded-claim rate

100 × checkable claims supported by retrieved evidence / all checkable claims in evaluated answers. Judge the cited passage, not the mere presence of a citation. Track answers with no checkable claims separately.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
The fraction measures evidence support under a rubric, not universal factual accuracy.
Investigate
Inspect source validity, chunk boundaries and retrieval versions. Sample judge disagreements for human review.
Rollback condition
Revert the corpus, retriever or prompt only if a replay isolates that component and the restored corpus remains valid and authorized.
Evidence
S1 · S6. Definitions and thresholds are this planner's design choices.
Evaluation

Retrieval recall@k

Mean per-query percentage of labeled relevant documents found within the top k results, on the versioned retrieval evaluation set. Freeze k and relevance labels. Exclude queries without labeled relevant documents and report their count.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
A current corpus can still miss the document needed to answer the question.
Investigate
Separate missing source documents from retriever/reranker failures. Rerun against the previous index.
Rollback condition
Restore a compatible index or retriever only after the same relevance set improves and required document updates remain available.
Evidence
S1 · S6. Definitions and thresholds are this planner's design choices.
Release / event

Corpus update lag

Maximum elapsed time from an authoritative source revision to the matching revision becoming queryable. Include outstanding updates using current time. Measure deletion propagation separately.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
An ingestion backlog can make grounded answers obsolete even when the model has not changed.
Investigate
Inspect failed ingestion, change feeds, index jobs and outstanding deletions. Identify affected sources.
Rollback condition
Pause answers from affected sources or use a reviewed fallback. Never restore revoked, deleted or unauthorized documents to improve a metric.
Evidence
S1. Definitions and thresholds are this planner's design choices.
Release / event

Unreviewed prompt releases

Count of active prompt-template hashes that lack a successful regression evaluation and an approved release record. Prompt entropy here means uncontrolled template changes, not a Shannon entropy metric.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
Without a release record, changed behavior cannot be attributed or reproduced.
Investigate
Stop promotion and compare active hashes with the approved prompt registry.
Rollback condition
Restore the approved compatible prompt only after verifying tool permissions, model and schema compatibility. This event bypasses sample and persistence gates.
Evidence
S1. Definitions and thresholds are this planner's design choices.
Release / event

Provider sunset lead time

Days remaining until the provider-published retirement date for the exact deployed model snapshot. Unknown retirement dates are unknown, never zero or safe.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
A scheduled model retirement requires a tested migration path. It is a lifecycle event rather than a statistical drift score.
Investigate
Verify the provider notice and rehearse the replacement against the current eval set.
Rollback condition
Use a supported pre-evaluated fallback if the snapshot becomes unavailable. Do not roll back to a retired model. Check daily and on notices. No sample gate.
Evidence
S1. Definitions and thresholds are this planner's design choices.
Release / event

Unauthorized tool effects

Count of observed tool side effects outside the allowed action, resource or approval scope. Log denied attempts separately. This signal concerns effects that occurred.

Example setting. Calibrate before use.

Rationale, response and rollback
Why this signal
A single confirmed unauthorized effect requires incident handling regardless of traffic.
Investigate
Disable the affected write capability, preserve evidence and assess compensation for completed effects.
Rollback condition
Switch to a read-only or human-mediated path. Deployment rollback does not undo payments, messages or other external effects. Bypass sample and persistence gates.
Evidence
S1. Definitions and thresholds are this planner's design choices.

Calibrate before you alert

03 / VALIDATE

When model, prompt and retrieval changes share one release, a falling score gives little help with attribution. Pharos Production addresses that problem in its published production AI instrumentation method, which records versioned inference context and evaluation results. This planner makes the baseline, response owner and rollback condition explicit.

  1. Freeze a representative reference

    Record actual start and end timestamps, release IDs, traffic cohorts and exclusions when implementing the plan. The baseline-days field is a duration, not proof that a valid dataset exists. Include relevant weekly or seasonal patterns. A rolling baseline can normalize a slow regression, so preserve a fixed accepted release for comparison.

  2. Measure outcomes alongside drift

    Input drift is a distribution change. Concept drift changes the relationship between inputs and the desired answer. A stable input mix does not rule it out. Compare task success and user outcomes using consistent labels. Split by model version, task, language and other relevant cohorts so a global average does not hide a localized failure. [S2]

  3. Backtest thresholds on your own history

    Replay stable windows and known incidents using the same eligible populations. Choose the minimum sample count from the precision your decision needs. Examine confidence intervals and the cost of false alarms. Persistence alone is not a significance test. Compare non-overlapping windows where possible and account for repeated testing across many signals. A drift-method threshold is tied to that method and cannot be copied into a different distance or test. [S3]

  4. Make missing data a separate condition

    A zero denominator means no rate, not a passing rate. Track telemetry completeness, unscored items, dropped traces, pipeline failures and late labels. If a cohort has too little data, extend its window or increase the appropriate evaluation sample. Do not widen the window for an urgent safety event simply to reach the sample floor. [S3] [S4]

  5. Rehearse the response

    Assign a real owner and a tested escalation route. Link each alert to the failing cohort, baseline, release and example traces. Diagnose whether the cause is ingestion, retrieval, a model change, the prompt or the evaluator itself. Retraining is one possible remedy. Repair a stale index or broken label pipeline first. Test the fallback in a canary and define recovery criteria before closing an incident.

Worked example: sampling is the bottleneck

For 1,000 requests per day, a 24-hour window and a 10% evaluation sample, the estimated budget is 100 evaluated requests per window. With a minimum of 100, there is no margin for ineligible requests or missing labels. At 200 requests per day, the same settings yield only 20 evaluations. The planner flags that gap. It does not infer that quality is acceptable.

evaluations = daily requests × window hours / 24 × evaluation percent / 100

A 14-day baseline under the first scenario would budget 1,400 evaluations if volume and sampling remained constant. It still needs a representative population, complete labels and a frozen rubric. Two consecutive 24-hour windows are a persistence policy for statistical signals. Critical events bypass it.

Keep telemetry attributable and limited

Correlate end-to-end application requests with model, retrieval and tool spans. Preserve model snapshot, prompt-template hash, corpus/index revision, evaluation-set version, judge/rubric version and trace ID. Model response time is one component of application latency. Retrieval, queues, retries and tool calls also contribute.

OpenTelemetry's GenAI conventions now live in a dedicated repository. Check the pinned source [S5] and each convention's stability before integrating an SDK. A tracing convention does not supply a grounding rubric, a validated threshold or an operational owner. Token usage can help explain workload shifts and cost, but a token change alone does not establish quality drift.

Collect only the content needed for the monitoring purpose. Raw prompts, responses and retrieved passages can contain personal or confidential information. Apply access controls, redaction and retention limits. Hashes are not automatically anonymous. Keep high-cardinality trace IDs and content out of metric labels. This page collects none of that telemetry.

Choose a response based on evidence, not the name of the alert
Observed conditionFirst actionRollback boundary
Input mix changes. Quality holdsReview new intent coverageNo rollback from a drift score alone
New release fails a matched evalIsolate component and verify fallbackRestore a compatible accepted bundle
Corpus is stale or access is revokedRepair ingestion or pause affected answersNever reintroduce invalid documents
Unauthorized side effect occursDisable affected write capabilityCode rollback cannot reverse external effects

Sources, scope and review

Source check: . Sources support the monitoring concepts. Metric formulas, sample estimates and example thresholds are transparent design choices for this worksheet. S1 is the contextual production engineering link above; S7 identifies the publisher. Neither is independent evidence for a business result.

  1. AWS: detecting drift in production applications

    Data/concept distinction, reference samples and investigation after a drift signal.

  2. Evidently: data drift

    Reference/current datasets, method dependence and separate null-rate checks.

  3. Arize: drift tracing

    Reference selection, comparison of production periods and delayed ground-truth labels.

  4. OpenTelemetry GenAI conventions

    Current documentation repository, pinned at source check. Individual conventions have their own stability status.

  5. Evidently: monitoring overview

    Batch evaluation and monitoring pipelines.

Publisher
Pharos Production
Implementation and technical review
Codex, automated source, code, calculation and browser checks for version 1.0.0. No independent human validation or production calibration is claimed.
Methodology
Primary-document review, explicit metric definitions and deterministic plan generation. No customer telemetry, empirical benchmark or deployed monitor was used.
AI disclosure
AI-assisted research, writing and implementation.
Scope limits
Planning software for chat, RAG and tool-using agents. It does not assess legal compliance or certify safety. Tool agents with retrieval should also include the RAG signals in their implementation.
Data handling
No analytics, cookies, browser storage or external API calls. GitHub Pages handles ordinary hosting requests under its own policies. Downloads are created locally. Exported owner and release fields contain what you entered.
License and corrections
Original code and planner content are MIT licensed. Linked source material retains its owners' rights. Report a formula or source error through repository issues. Do not include confidential traces.

Metric catalog JSON · Source register JSON · Example plan JSON · Changelog