Build vs Buy AI Agent Scorecard: 12 Weighted Criteria

The score is a decision trace, not a verdict machine

A useful sourcing decision shows why each answer was chosen, which facts remain unknown and what would change the outcome. This model was prepared by the Pharos Production AI engineering team as a transparent decision aid, not a product recommendation or a shortcut around discovery.

The number has one job: make competing assumptions visible. It is decision support, not a probability, forecast, confidence score, success rate or financial recommendation. If the evidence gate fails, the tool returns INSUFFICIENT_DECISION_EVIDENCE instead of converting missing facts into false neutrality.

Privacy boundary: calculations run in your browser. This static site has no account, backend, analytics, cookies, local storage or query-string state. Refreshing the page clears the current answers.

Interactive decision record

Score 12 AI agent sourcing criteria

Choose the statement that current evidence supports. Use Unknown when the evidence is missing. The model will tell you which unanswered constraint blocks readiness.

01 Strategic differentiation Weight 3

How much of the agent's behavior creates a durable product or operating advantage?

Useful but commodity Several credible products cover the capability. Some differentiated logic The shell is common, but domain rules matter. Core proprietary capability Behavior or feedback is part of the moat. Unknown The strategic role has not been agreed.

02 Workflow and policy uniqueness Weight 3

Are the agent's rules, tools and exception paths standard or organization-specific?

Standard workflow Common steps and limited exceptions. Standard core, custom edges A few rules or actions need owned extensions. Proprietary policy engine Custom rules and exceptions shape the result. Unknown The real workflow has not been mapped.

03 Data control and residency Weight 3 / critical gate

Can an approved vendor path meet location, retention, deletion and training-use constraints?

Approved vendor path exists Documented handling requirements are met. Sensitive boundary can be isolated Only approved fields cross the managed layer. Controlled custom path required The current shortlist fails a mandatory constraint. Unknown Classification or vendor handling is unresolved.

04 Integration depth Weight 3

How deeply must the agent interact with systems, permissions and failure recovery?

Standard connectors and reads Supported objects and limited actions. Mixed connector depth Common systems fit; crown-jewel adapters do not. Deep custom transaction paths Legacy access, writes or custom recovery matter. Unknown The integration inventory is incomplete.

05 Time to validated value Weight 3

How soon must a production-like workflow produce measurable evidence?

Evidence needed immediately A configurable baseline must arrive first. Phased evidence window Baseline now, owned boundary after validation. Longer validation horizon The roadmap supports iterative architecture. Unknown No deadline or measurable outcome is agreed.

06 Engineering and product capability Weight 3 / critical gate

Can a named internal team design, test and evolve the agent beyond a prototype?

No dedicated build team Durable cross-functional ownership is unavailable. Strong software team, selective gaps Own domain work; buy platform gaps. Experienced agent delivery team Named owners have relevant delivery experience. Unknown Staffing is hypothetical.

07 Lifecycle ownership Weight 3 / critical gate

Who will operate evaluation, monitoring, incidents and model or vendor changes?

Need managed lifecycle coverage A vendor must carry much of the platform work. Shared operating model The vendor runs layers; you own domain outcomes. Full internal lifecycle team Releases, regressions and incidents have owners. Unknown No operating owner accepted responsibility.

08 Budget and capital shape Weight 2

Does the organization prefer lower initial commitment or controlled long-term investment?

Lower initial commitment Recurring spend is easier to approve. Balanced spend profile Fund a platform and bounded custom work. Front-loaded investment is approved Team and contingency are funded. Unknown Budget shape is not approved.

09 Scale and unit economics Weight 2

Are demand, workload units and marginal costs measurable enough to compare paths?

Demand is uncertain or modest Avoid fixed platform investment for now. Scale is measurable, crossover unclear A live baseline is needed. High predictable scale Owned cost controls can compound. Unknown No workload unit or demand model exists.

10 Portability and vendor dependency Weight 2

What must remain movable if the model, platform or commercial terms change?

Contractual dependency is acceptable Exit terms bound the managed dependency. Portable core, managed shell Data, domain logic and evals remain movable. High portability requirement Runtime boundaries and redeployment are controlled. Unknown Exit requirements are undefined.

11 Evaluation and observability control Weight 2

Can a bought product expose the evidence needed to test domain quality and actions?

Vendor evidence is sufficient Required traces, eval hooks and exports exist. Own evals around a bought runtime Tests and release gates remain yours. Domain-specific evidence is central Custom traces or controls are required. Unknown The evidence contract is not defined.

12 Compliance and third-party path Weight 2 / critical gate

Which sourcing path can produce the required approvals, records and accountability?

Approved vendor controls fit Procurement and control gates can be met. Control boundary must be shared Evidence and obligations split by owner. Custom controlled path required The current vendor shortlist fails a gate. Unknown Obligations or approvals are unresolved.

Decision record

Complete the scorecard to produce a result

The initial state is intentionally unresolved. Choose a preset or answer each criterion, then calculate.

Evidence status: NOT_CALCULATED
Outcome: UNRESOLVED
Decision index: -
Known-weight coverage: 0%

No decision has been calculated.

Open formula

How the 31-point signed model works

Each answer is -1 for buy, 0 for hybrid, +1 for build or null for unknown. Multiply that value by the criterion weight, sum known points and normalize only across known weight:

decision index = round(100 x signed known points / known weight)

Buy is -30 or lower. Build is +30 or higher. The interval between them is hybrid. Results within seven index points of either threshold carry a boundary warning because a small evidence change can move the decision.

Readiness is separate from direction

Known-weight coverage must reach 75%.
Data control, team capability, lifecycle ownership and compliance path must be known.
A constraint that blocks the numeric outcome triggers review.
Unknown never becomes a hidden zero.

The thresholds are versioned editorial defaults. They have not been statistically calibrated against project outcomes.

Compressed comparison

Build, buy and hybrid AI agent paths

The table states the ownership consequence of each path. It does not assume that one path is mature, secure or economical without evidence.

Operational differences among build, buy and hybrid sourcing paths
Decision dimension	Build	Buy	Hybrid
Differentiated logic	Owned end to end	Constrained by product configuration	Owned domain layer over managed components
Time to first evidence	Depends on staffing and integration	Can be faster when a credible product fits	Managed baseline followed by owned extensions
Data boundary	Designed and operated internally	Defined by vendor architecture and contract	Sensitive zone isolated from replaceable services
Integration control	Custom depth and recovery semantics	Supported connector surface	Buy the long tail; own crown-jewel adapters
Evaluation	Full custom harness and traces	Vendor evidence plus customer validation	Organization-owned tests around a managed runtime
Lifecycle load	Highest internal platform responsibility	Vendor operates more of the platform	Explicitly divided by component and outcome
Vendor dependency	Still exists at model, cloud or framework layers	Highest at runtime and commercial layers	Bounded by portable data, logic, evals and adapters
Economic pattern	Front-loaded work plus internal recurring cost	Lower setup can become recurring usage exposure	Mixed setup and recurring cost with boundary overhead

Decision surface

Direction, evidence and constraints are three different axes

A build-leaning index is not ready when critical evidence is absent. A buy-leaning index is not actionable when a mandatory data or compliance constraint blocks the vendor path. The decision record therefore evaluates direction first, readiness second and blocking constraints third.

Compute direction from known weighted answers.
Withhold the decision if evidence coverage fails.
Override a conflicting shortcut with constraint review.
Pair the surviving path with TCO and an exit test.

A decision index from buy through hybrid to build passes through an evidence gate and a constraint review before a sourcing record is ready. — Textual mirror: direction does not become a decision until evidence and mandatory constraints are checked. Original diagram, Pharos Production, 2026.

A hybrid AI agent stack keeps domain policy, sensitive data, evaluation assets and crown-jewel adapters inside an owned boundary while buying models, standard connectors and commodity operations. — Hybrid is a component boundary, not a vague middle score. Original diagram, Pharos Production, 2026.

Architecture boundary

Hybrid means owning the parts that preserve differentiation and control

The criteria extend the build vs buy AI agent decision framework with explicit unknown handling, boundary warnings and exportable evidence. The most useful hybrid design assigns ownership by component instead of splitting the project by percentage.

Usually own: Domain policy, approval rules, sensitive data boundaries, evaluation sets, release gates and crown-jewel adapters.
Often buy: Foundation model access, standard connectors, baseline administration, commodity observability and replaceable infrastructure.
Contract explicitly: Data use, exports, logs, termination help, deletion, service levels, change notices and support boundaries.
Test before approval: Identity fidelity, action authorization, failure recovery, trace export and redeployment after simulated vendor exit.

Editable cost model

Compare three-year total cost of ownership

The worked scenario is illustrative. It is not Pharos pricing, a vendor quote or a market benchmark. Replace every amount with your own approved labor, platform, usage, control and exit assumptions.

Editable USD assumptions by sourcing path and cost cadence
Cost category	Cadence	Build	Buy	Hybrid
Discovery and architecture	One-time
Implementation and integration	One-time
Data preparation and migration	One-time
Licenses, models and infrastructure	Annual
Internal product and operations team	Annual
Evaluation, security and compliance	Annual
Change, training and support	Annual
Exit and switching reserve	End of horizon

Illustrative three-year result

Calculate to refresh the totals after editing any amount.

Build

$2,100,000

One-time: $470,000
Annual: $530,000
Exit: $40,000

Buy

$1,990,000

One-time: $210,000
Annual: $520,000
Exit: $220,000

Hybrid

$1,940,000

One-time: $340,000
Annual: $490,000
Exit: $130,000

Formula: one-time total + (annual total x 3) + exit reserve. Lowest modeled cost does not override mandatory constraints or prove the best outcome.

Cost cadence

Three-year TCO must keep recurring ownership visible

A setup quote and an annual subscription answer different questions. The model groups discovery, implementation and migration as one-time costs; it multiplies infrastructure, internal operations, evaluation and support by three; then it adds an explicit exit reserve.

For a meaningful comparison, attach a measurable unit such as cost per successful task, per resolved case, per active user or per approved action. That lets the team distinguish growth from waste and compare architecture changes with business value.

A three-year cost timeline showing one-time discovery and implementation, recurring annual platform and operating costs, then an exit and switching reserve. — Textual mirror: TCO equals the one-time layer plus three annual layers plus an exit reserve. Original diagram, Pharos Production, 2026.

Decision scenarios

When build, buy and hybrid become defensible

These scenarios illustrate evidence patterns. They are not client cases, outcome claims or substitutes for your own data.

Buy: a common internal workflow

Buy becomes defensible when credible products cover the workflow, data handling is approved, connector depth is standard and a measurable baseline is needed before a custom team could be staffed.

Proof to collect

Production-like connector test
Data-flow and deletion evidence
Usage and renewal cost bands
Exit export and redeployment test

Build: a controlled differentiated system

Build becomes defensible when proprietary policy or feedback creates durable advantage, the vendor shortlist fails a binding constraint and a named team can own evaluation, operations and incidents.

Proof to collect

Architecture and trust boundary
Named owner capacity
Domain evaluation set
Three-year staffing and operations budget

Hybrid: own the control spine

Hybrid becomes defensible when models, connectors or administration are replaceable but domain policy, sensitive data, evals and crown-jewel integrations must remain portable and organization-owned.

Proof to collect

Component ownership map
Shared-control RACI
Portable prompt, state and eval assets
Migration gate after a live baseline

Vendor diligence

Test what the vendor path actually owns

Data: trace input, state, logs, training use, retention, deletion and subprocessors.
Identity: verify SSO, role mapping, delegated authorization and service-account boundaries.
Actions: test approval, idempotency, rollback, reconciliation and immutable audit evidence.
Quality: run your domain set and export traces, failures, costs and version identifiers.
Commercials: model seats, usage, support, overages, renewal, price changes and termination help.
Exit: export workflows, prompts, data, evals, logs and configuration, then redeploy a critical slice.

Build diligence

Prove the internal path can be operated

Product: name the outcome, owner, affected users, wrong-action cost and stop condition.
Platform: assign runtime, data, identity, observability, deployment and on-call ownership.
Evaluation: version test data, failure labels, release gates and human-review thresholds.
Security: map tool permissions, secret handling, egress, memory and supply-chain controls.
Maintenance: budget model changes, API churn, drift, data repair, user support and incidents.
Opportunity cost: identify the roadmap work displaced by platform ownership.

Failure modes

Six ways teams manufacture a confident answer

Importance becomes differentiation

A workflow can be important without being proprietary. Ask what a competitor cannot reproduce by purchasing the same platform.

Unknown becomes neutral

Missing data is assigned a middle score and quietly increases certainty. This tool removes unknown weight and can block readiness.

Demo fit becomes production fit

A polished prompt hides permissions, failure recovery, auditability, support and real data. Test the hardest production slice.

Code ownership becomes portability

Owning source code is incomplete if state, prompts, evals, logs or deployment remain trapped in a vendor runtime.

Setup cost becomes TCO

Labor, evaluation, security, operations, support, downtime and exit work disappear from the comparison. Keep cadence visible.

Platform operation has no owner

A build proposal names developers but not product, data, security, release, incident and on-call owners. Treat that as a blocked build.

Method, evidence and E-E-A-T

What is sourced, what is authored and what remains unverified

The model separates external evidence from editorial decisions. Current sources support the relevance of lifecycle risk, TCO breadth, readiness and recurring make-or-buy factors. No cited source validates the 31-point weights, -30/+30 thresholds, seven-point boundary or illustrative dollar amounts.

Publisher: Pharos Production Editorial Team
Model version: 1.0.0
Published and verified: August 25, 2026
Independent review: Independent technical review was not performed for this release.
Drafting disclosure: AI-assisted drafting was used; the release was source-checked and mechanically tested.
Update policy: Review source status and model assumptions at least quarterly or after a material framework, regulation or vendor-market change.
Correction policy: Document material corrections in the public changelog, update the model version when behavior changes and retain source boundaries.
Scope boundary: Architecture and procurement decision support only; not legal, security, compliance or financial advice.

Authoritative and research sources

NIST AI Risk Management Framework Voluntary full-lifecycle guidance; v1.0 is under revision.
NIST AI 600-1 Generative AI Profile Cross-sectoral risk actions; no sourcing thresholds.
FinOps Foundation terminology TCO and unit-economics definitions; no AI-agent pricing.
KPMG build-buy-borrow paper Consultancy-authored readiness framework with a commercial interest.
Klotz, The Buy-or-Build Decision, Revisited Conceptual preprint, not an empirical calibration study.

Open artifacts

Questions the model leaves open

Build vs buy AI agent FAQ

Why does an unknown answer not count as hybrid?

Hybrid is a sourcing choice supported by evidence. Unknown means the evidence is absent, so the model removes that criterion's weight and can withhold a decision.

Does the lowest three-year TCO automatically win?

No. TCO is one decision record. A cheaper path can still fail a mandatory data, security, compliance, integration or operating constraint.

Can this scorecard select a specific AI agent vendor?

No. It selects a sourcing posture. Specific vendors still require feature validation, data-flow review, contract analysis, security testing and an exit test.

When should the build-versus-buy decision be rerun?

Rerun it after a production-like proof, a material pricing or contract change, a new regulatory constraint, an architecture change or a shift in demand and staffing.

Does buying an AI agent transfer accountability to the vendor?

No. A vendor can operate part of the stack, but the organization still owns use-case approval, data access, outcome monitoring, human oversight and incident decisions.