Evals as acceptance criteria: building quality into the build process

In the first module, you made architecture decisions: patterns, integration points, and how your system should respond to different inputs. What you don't have yet is confidence that those decisions hold when the inputs are real and the users are unpredictable.

This is where evals come into the picture. Evals are short for evaluations, and they let you test your system's behavior before it goes to production or after model updates, so you discover problems before they occur. This section explains what evals are, why they matter, and how to use them to get ahead of issues before your users find them for you.

Evals before code: why the order matters

An eval, or evaluation, is a structured test that checks whether your system returns the expected and accurate output. Although it sounds straightforward, the timing is extremely important. The standard approach is to build the system, see if it looks right, and conduct tests later. This isn't always the best approach.

Writing your eval suite before you write production code forces three things to happen that are otherwise easy to defer:

• First, state what success means in measurable terms
• Second, expose design assumptions early, when changing them is still cheap
• Third, give yourself a gate that can determine whether a model swap, a prompt change, or a new retrieval strategy measurably improved the system

An eval suite belongs at the beginning of your build, defined before production code is written, rather than at the end as a QA step. In fact, if you cannot write an eval for behavior, then you have no reliable way to measure whether that behavior is present. This means that every change you make to the system isn't verifiable. Adding an eval suite at the beginning allows you to verify throughout the entire build.

How the eval workflow runs: from task definition to result

A well-constructed eval workflow runs sequentially through the stages below. Each stage produces an artifact that feeds into the next stage:

Model-based vs. code-based evals: when to use each

Not every behavior you need to evaluate can be checked the same way. Some behaviors have a single correct answer: the output is either a valid JSON, or it isn't. A second category is whether the output matches the expected tone or style of language. These two categories of behavior need different evaluation tools, and choosing the right one for a given behavior is important to ensure accuracy and save costs.

The three types of evals have different speed-versus-flexibility tradeoffs:

• Code-based evals run deterministic checks in milliseconds and cost almost nothing.
• Model-based evals use a judge model to assess outputs that require interpretation and cost roughly as much as the model call itself.
• Human-review evals rely on human judgment for high-stakes or novel behaviors where neither code nor a model judge can be trusted to evaluate reliably. Human-review evals are the slowest and most expensive option.

The grading ladder: choosing how to grade

Not every behavior should be graded the same way, and the choice of grading method follows a deliberate ladder. Reach for the cheapest reliable method first and climb only when the behavior demands it.

• Code-based grading, wherever the behavior allows it. Deterministic checks, including schema validation, exact match, length, and presence, run in milliseconds, cost almost nothing, and never drift. If a behavior can be checked in code, then it should be.
• LLM-as-judge, when the behavior needs interpretation. Use a judge model for outputs that require judgment. Make the judging rigorous by using detailed rubrics, constrained verdicts (a small fixed set of labels rather than free-form scores), calibration against human-labeled examples, and grading with a different model than the one whose outputs you're evaluating, to avoid self-preference.
• Human grading, as the last resort. Reserve human review for high-stakes or novel behaviors where neither code nor a calibrated judge is trustworthy yet. It is the most expensive and least scalable option.

Favor volume over perfection. Many automatically gradable cases beat a handful of manually-graded ones: broad, cheap coverage catches more regressions than a small, painstaking set, and it can run on every change.

Defining success criteria: turning a business requirement into a measurable threshold

A business requirement like "summarize claims accurately" does not really tell you what to measure. The process of turning it into an eval criterion has the following steps:

• Identify the behavior specifically: "Summarize claims accurately" should be updated to "extract the filer's name, claim number, incident date, and claimed amount from each document."
• Set the threshold: Decide what counts as passing. If, for example, your thresholds are 100% accuracy on structured fields, less than 2% hallucination rate, and response within schema 99.5% of the time, those numbers should come from the business requirement. Don't just choose what your first prototype happens to achieve; find guidance on setting eval thresholds at platform.claude.com/docs/en/test-and-evaluate/develop-tests.
• Identify the failure modes: Continuing with the same example, an output that might not be acceptable can include a fake claim number, a missing incident date, or a value from the wrong claim. Each failure mode is a category in your eval dataset.
• Include adversarial inputs: It should include documents with missing fields, handwritten sections, unusual formatting, and non-standard layouts. If your golden dataset contains only clean inputs, then your eval scores will not predict production performance.

Evals as the gating mechanism for change

Every change to a production Claude system, whether it is a model swap, a prompt revision, a context strategy change, or a retrieval configuration update, should run through the eval suite during development before it moves to production. This is the only reliable way to know whether a change has improved the system.

A single-turn eval set will not tell you how the system holds up across a conversation. Multi-turn evals are a separate category that scores the system over a sequence of exchanges rather than on a single prompt and response. A multi-turn eval checks a few criteria: whether the system keeps prior context straight across turns, whether it answers a follow-up prompt without inventing details that were never said earlier in the conversation, and whether output quality holds as the conversation runs longer. Because the unit being scored is a whole conversation, this category needs its own golden dataset. This consists of full conversation transcripts with known high quality responses at each turn, covering the follow-ups, topic shifts, and conversation lengths the system will see in production.

Consider a team building a document summarization workflow that revised their summarization prompt but did not update their eval suite to match. The eval suite passed all required checks. Two days after the swap went to production, field reports showed that multi-clause legal sentences were being truncated into summaries. The root cause was that the eval set predated the prompt change and didn't match the behavior that changed.

In short, you should run evals before every change to keep the eval set current with the system it is measuring.

The eval suite that measured the wrong thing

A partner postmortem that needs to be reconstructed

The following is a composite postmortem representing a pattern that appears repeatedly in field deployments. The team built a contract review assistant for a professional services firm. They manually tested the assistant against ten contracts their team knew well, declared the system ready, and then moved to production. The regression arrived two weeks later.

The decisions that led us here

• The eval dataset was built from convenient inputs rather than a representative sample. An eval suite that does not cover the input distribution it will face in production is measuring a different system than the one you are shipping.
• The eval input set was not updated after the prompt was changed. The eval was still passing every time because it was assessing behavior the prompt no longer produced. The scores looked stable because nothing was testing what had changed.
• Manual spot-checks were treated as an eval substitute. Spot-checks can confirm that a specific input produces a specific output, but they cannot tell you whether the system behaves correctly on unknown inputs.

Sort the eval types

Eight evaluation tasks are listed below. Drag each one into the bucket where it belongs: model-based eval or code-based eval. The right placement follows whether the behavior being checked requires interpretation or is straightforward.

From POC to production: cost, latency, and reliability

Your eval suite tells you whether the system behaves correctly, but nothing about whether it can afford to behave correctly at the volume your partner expects. That gap is the Proof of Concept (POC)-to-production gap, and it has four dimensions: cost, latency, reliability, and failure modes. All four are invisible in a demo.

A POC also gives you the first signal on whether the system moves the business metric it was designed to improve. That signal is worth capturing deliberately: measure the outcome the partner cares about on your POC sample, even before you model production cost. A cost profile that fits the budget on a system that does not produce a measurable improvement in the metric that matters is still a failed deployment.

Where a Proof of Concept (POC) and a production system differ

A POC is designed to demonstrate capability. It runs at low volume, on clean inputs, and with a patient user. A production system runs at the volume your partner's business generates, on real inputs, and with users who have no tolerance for slow or incorrect responses. The four dimensions where a POC misleads you are cost, latency, reliability, and failure modes.

Cost and latency modeling: Know the numbers before you build

The cost and latency model is built before you finalize the architecture. The three inputs you need are call volume (requests per day or per month), token budget per request (input tokens plus expected output tokens), and model tier. From those three, you can estimate monthly cost and check it against your budget ceiling before any code is written.

The token budget per request is where most cost models go wrong. Teams calculate the average token count on the inputs they have and assume that is the distribution. In practice, token distributions are often skewed: most requests are short, but a tail of long requests consumes a disproportionate share of the total cost. A cost model based on average usage can significantly underestimate the cost impact of these longer requests, often by a factor of two or three.

Latency follows a similar pattern. Median latency reflects the middle of the pack, but SLA breaches are usually caused by slower requests at the high end. That is why p95 is a more useful design target than the median. P95 is the latency value below which 95% of requests complete. Only the slowest 5% fall above it.

Caching is the most effective cost and latency lever when the system prompt is long and stable. Prompt caching preserves the processed prompt prefix for the cached tokens, so the API does not reprocess them on subsequent requests. Savings scale with both the length of the cached prefix and how often it's reused. If, for example, cache reads are charged at 10% of the standard input token rate, a long prefix reused across many requests produces the largest effective savings. Find the most current cache read rate at platform.claude.com/docs/en/about-claude/pricing. The risk is consistency: if the cached content needs to reflect live state, caching creates a consistency window that may violate the requirements of the use case. What is stored is the prompt prefix.

Reliability controls: what to build into every model call

The reliability controls that belong in the production Claude system address different failure scenarios and sit at different layers of the call stack.

• Transient error recovery with exponential backoff. When a model returns a transient error, such as a rate limit 429, timeout, or 5xx, the system should retry with progressively longer delays between attempts. This prevents a flood of retries from turning a brief hiccup into a prolonged outage. Set the maximum number of attempts and total wait time based on how much delay your use case can tolerate.
• Fallback chains. If the primary model or endpoint is unavailable, the system should automatically route the request to an alternative such as a different model tier or a cached response. It should not raise an error to the user. Fallback behavior should be tested as part of your eval suite.
• Circuit breakers. A circuit breaker measures the error rate on a downstream dependency and trips when errors exceed an established threshold. Once tripped, requests fail immediately rather than waiting for a timeout. This prevents one degraded dependency from taking down the broader system.

Reliability controls must sit at the right stage to be effective: new attempts belong close to the API call, circuit breakers at the service boundary, and fallback chains in the orchestration layer. Placing them in the wrong layer means protecting the wrong part of the system and leaving the right part exposed.

Failure modes by architecture type

Agents handle tasks that cannot be completed in a single model call: they can use tools, observe results, adjust plans mid-execution, and complete multi-step processes that require dynamic reasoning at each step. Claude Code is a production example, and it navigates codebases, runs tests, applies fixes, and iterates across a workflow that is impossible in a single-turn architecture. The controls below govern how to build these systems reliably.

The demo cost profile that became the production bill

Three quotes from post-launch review

The quotes below are from a single team's post-deployment review, 60 days after moving a document triage system to production. Each quote identifies a different aspect of the same mistake.

What broke and why

Each quote names a distinct failure, but they compound in order. The cost model was wrong because it was built at the incorrect volume. The token distribution assumption was wrong because it was built on the incorrect inputs. The reliability failure was invisible in development because failure cases were never tested.

All three failures share the same root cause: the POC was treated as a cost and reliability model, not just a capability demonstration. A POC answers the question "can the system do this", but it does not answer "what does it cost to do this at scale" or "what happens when a dependency fails."

Cost & reliability calculator

Use the calculator to explore configurations. Adjust the model tier, prompt caching, max_tokens cap, and call-volume multiplier; the readouts show monthly cost and p95 latency for each setting. Your goal: find a configuration that meets both the cost ceiling and the latency target at the same time. The calculator only displays values; it does not score your exploration. (Figures are representative for modeling; confirm current rates at publish time.)

Decision: Once you have a configuration that meets both targets, which lever did the most work to get there?

Use-case sizing and feasibility

The production readiness checklist tells you what a system must achieve to be viable. It covers both the quality of the model's outputs and the reliability of the system around it. Output quality is validated through evals, and system reliability is validated through architecture controls like retries, fallbacks, and circuit breakers. Meeting both bars is what production readiness means.

Sizing tells you whether a specific business problem can meet that bar, and what constraints govern the design. Feasibility fits into one of three states: feasible as scoped, feasible with constraints, and not feasible. Identifying the state correctly is what makes a scoping document useful.

How to size a use case

Sizing a use case means producing a cost model before any code is written. The model does not have to be precise, but it must be accurate enough to validate the architecture against the budget and surface token distribution assumptions before they are formalized.

Four inputs drive the model: call volume, token budget per request, model tier, and sensitivity parameters.

• Step 1: Estimate call volume. How many requests are made per day or per month? This number comes from the business requirement, not from the developer's intuition. A customer service agent that handles 1,000 conversations per day produces 1,000 Claude calls per day, plus any multi-turn continuation calls. Get this number from the business owner. A sample dataset will not give you an accurate figure.
• Step 2: Set the token budget per request. The token budget has two components: input tokens (system prompt, retrieved context, and user message) and output tokens (expected response length). Model the distribution rather than just the average. If document lengths vary widely, the cost model should account for the typical cases as well as the extremes. If the system prompt is long and stable, prompt caching can meaningfully reduce input costs. Caching requires explicit cache_control markers in the request. Cache writes incur a higher per-token cost than standard input, so the cost model must account for the write cost on first use. The default cache TTL is 5 minutes; workloads with request frequency lower than TTL will not realize consistent caching savings.
• Step 3: Project the monthly cost. Multiply call volume by the input token count at the input token rate. Separately, multiply the output token count at the output token rate. Then, add both figures. Input and output tokens are priced at different rates on all model tiers. If prompt caching applies, use the cache read rate for cached input tokens, not the standard input rate. Verify current rates at platform.claude.com/docs/en/about-claude/pricing before finalizing the model. Add caching savings if applicable. Compare the result to the cost ceiling from the production readiness checklist. If the projection exceeds the ceiling, the architecture needs to change before a line of code is written. If, for example, Batch API provides a 50% price reduction relative to standard API pricing and supports up to 100,000 requests per batch, model it as a cost alternative for any workload where the SLA permits asynchronous processing. For regulated workloads, verify whether batch processing is covered under the partner's BAA and compliance configuration before routing PHI or similarly governed data through it. Find the most current Batch API discount rate and batch size limit at platform.claude.com/docs/en/about-claude/pricing.
• Step 4: Run sensitivity analysis. What happens to cost if call volume doubles? What if the token distribution shifts toward the tail? Sensitivity analysis tells you how fragile the cost model is and where the assumptions need to be verified with the business owner before committing to the design.

How to scope a use case

The discovery sequence for turning a business requirement into a scoped architecture runs in four steps. Skipping any step produces a commitment that will not survive the next conversation with the business owner.

• Step 1: Business requirement to capability list. What does the system need to do? Name each capability separately. "Process insurance claims" is a goal. The capabilities might include extracting structured fields from the claim document, looking up policy coverage from the policy database, routing the claim to the appropriate adjuster queue based on claim type and value, and drafting the adjuster notification. Identify them separately so you can assign each to the appropriate owner.
• Step 2: Capability list to architecture sketch. For each capability, decide where it belongs. Which capabilities does Claude own? Which belong to existing systems? Which require a human in the loop? This is the decomposition step from Module 1, applied to a specific use case.
• Step 3: Architecture sketch to boundary conditions. State the conditions under which the architecture works and the conditions under which it does not. Feasibility is a verdict plus the constraints that make the verdict true. An architecture that works for documents up to 20 pages but fails for longer documents has a boundary condition that must be documented.
• Step 4: Boundary conditions to scope in the SOW. The statement of work contains the boundary conditions. This ensures that the development team and the business owner both understand what the system is designed to handle and what it is explicitly out of scope.

How to conduct a technical feasibility assessment

A feasibility assessment that only asks "can Claude do this" is a capability check. The four AI properties give you a structured way to identify where the design will require compensating controls, and what those controls should be.

Select each property to see the feasibility question it raises and where the design compensates.

Feasibility verdicts

Once the scoping sequence and technical assessment are complete, the architecture is ready for a feasibility assessment. There are three possible outcomes.

Business value and ROI mapping: turning a feasible design into a justified investment

A feasibility verdict tells the business owner that the system can be built within the budget and the constraints. It does not tell them whether building it is worth doing. Determining business value and ROI mapping are the steps that answer that second question. They connect the scoped architecture to the financial and operational outcomes the business expects, expressed in terms the business owner already uses: hours saved, error rates reduced, cycle time shortened, or revenue protected. The mapping turns a technical design into a decision a budget holder can defend.

The business case rests on five main pillars, and identifying them keeps the ROI conversation in the language the blueprint and the business owner both use: efficiency (the same work done faster or cheaper), transformation (work that was not feasible before becoming possible), productivity (more output from the same people), solution cost (the run cost of the system itself), and performance SLAs (the service levels the deployment must hold). Map each ROI claim to the pillar it advances so the value statement captures both the number and the kind of value it represents.

The mechanism is a comparison between two states. The baseline state is how the work is done today, measured in the unit the business cares about. The projected state is how the work is done once Claude is in the workflow, measured in the same unit. The value is the difference between the two states, minus the cost of running the system. The cost figure comes directly from the sizing model produced earlier in this cluster, so the ROI calculation reuses work you have already done rather than starting over.

The mapping is built in four steps, and each step must be grounded in a number the business owner will recognize:

• Step 1: Name the baseline in a business unit. Start from how the task is performed today and measure it in the unit the business already tracks. For a claims review workflow that is the analyst hours per claim or the average days to resolution. The baseline must come from the business owner's own operational data, because every later number is compared against it. A baseline pulled from intuition produces an ROI figure no finance team will accept.
• Step 2: Predict the post-deployment state in the same unit. Estimate how the same task performs once Claude is in the workflow, measured in the identical unit as the baseline. Where the feasibility verdict requires human review, the projection must include that cost. Routing low-confidence output to a reviewer reduces labor, but it does not eliminate it entirely. When the design specifies human-in-the-loop review, projecting full automation overstates the value and produces a number operations will reject.
• Step 3: Subtract the run cost from the sizing model. Take the projected monthly cost produced during sizing and treat it as the recurring cost of the new state. The value of the deployment is the operational gain from Step 2 minus this run cost. This step isolates recurring run cost only. Build cost is treated separately and is accounted for in the payback period calculation in Step 4. Including the sizing output in the ROI calculation keeps the two analyses consistent. A change to the token budget or model tier then updates both the cost ceiling and the value case together.
• Step 4: State the payback period and the sensitivity. Express the result as a payback period, which is the time it takes for the accumulated operational gain to cover the build cost and the run cost. Then state how that period moves if the volume assumptions or the gain-per-task assumptions are wrong. Quoting a payback period without sensitivity analysis invites decisions based on a single optimistic scenario, one that often fails when the business case meets real volumes after launch.

The output of the mapping is a short value statement the Architect hands to the business owner alongside the feasibility verdict. It ties the answers to "Can we build this?" and "Is it worth building?" back to numbers the business already owns. The two artifacts travel together into the statement of work.

These are the most common ROI map errors, what causes them, and where they tend to appear.

The scoping call that skipped the constraints

A scoping conversation: Commitment made too early

The following is an extract of a pattern that surfaces in discovery calls when the capability question is answered before the constraint questions are asked.

What went wrong

The feasibility verdict was issued before three essential constraints were gathered: call volume (800 per day), input size (up to 300 pages), and latency requirement (30 seconds).

Whether a lengthy contract fits within the context window depends on the model tier selected. Models with a 1 million token context window can handle a 300-page contract without chunking; a model with a 200k token window may require a chunking strategy for the longest documents. Context window capacity is therefore part of the model tier decision, not a settled assumption.

At 800 requests per day, the 30-second latency requirement is not as limiting as it seems. This averages out to one request every 108 seconds. Sequential processing is viable at that volume without running requests in parallel. At that request rate, volume pressures cost, not latency. Latency is driven by task complexity, model size, and output length. Those are the variables the model tier decision needs to be built around.

The architect confirmed capability as a necessary first step, but it was also the last step, which meant the commitment was made before the design was possible.

Justify the feasibility call

For each scenario, choose the option that names both the correct feasibility verdict and the single load-bearing constraint behind it. The verdict alone is not enough, the constraint is what makes the verdict defensible.

Enterprise integration patterns: identity, auth, data, and observability 

Compliance constraints eliminate entry point options before any other decision is made, so entry point selection comes first. The remaining five layers govern how the integration is built from there.

Sizing tells you what the system needs to do and whether it can do it within the constraints. Integration patterns tell you how it connects to the enterprise stack. That connection has five layers, and each layer has an architectural decision that belongs to the Architect, as opposed to the implementation team.

Entry point selection: which integration to use and when

The first decision in any integration is which entry point the system will connect through. Compliance constraints eliminate options at this first stage, before any other architectural decisions are made. The table covers the five available entry points, when each one applies, and what each one costs in flexibility or maintenance.

Security and compliance constraints at the integration layer

Module 1 established that regulatory and policy constraints, laws like HIPAA, GDPR, and FedRAMP, attorney-client privilege, and data-residency requirements eliminate entry point options before any other decisions are made. This section applies the same logic one level deeper: which integration pattern on that entry point may, with appropriate architectural choices, internal policies, contractual terms, and other items, help satisfy or otherwise mitigate concerns about the constraint.

The table below extends that analysis from entry point selection to full integration design.

Note: This should not be seen as legal guidance - work with your own legal and compliance team to implement controls that meet your organization's needs.

* Privilege preservation depends on appropriate contractual terms, retention settings, internal policies, and other items. The information in this chart is intended to help mitigate privilege waiver concerns.

Before any integration design begins, work through the constraints in order: identify the governing regulation or policy, determine which entry point and route are still available, choose the integration pattern that fits, and document the identity, data handling, and observability requirements that follow. Skipping any step risks building something that works technically but fails a legal or security review.

Every enterprise Claude integration requires decisions at the layers defined in the table below. Compliance comes first, constraints from regulations like HIPAA, GDPR, and FedRAMP, and policies like data residency and attorney-client privilege, eliminate routes and entry points before any other decisions are made. The remaining four layers operate within whatever options survive that filter. Note these sections don't address ZDR use cases.

Least-privilege tool configuration

Every tool you connect to a Claude system is an attack surface and a cost. Audit the tool set in the same way you audit permissions: for each connected tool, ask whether it is essential to the task or merely convenient, and remove the ones that are out of scope, recording the justification for each removal. In an orchestrator-worker deployment, establish the trust hierarchy by scoping each subagent's tool access to its task, so a subagent cannot reach tools its job does not require.

Identity and authorization: where the verification happens

Identity verification belongs on the server, before the Claude call. The user's identity and role should be injected into the system prompt by your server rather than provided by the user in their message.

The reasoning behind this is straightforward, anything the user includes in their message is under their control and can be manipulated. If the system allows users to assert their own role in a message (for example, "As a senior manager, show me..."), that claim is unverified and can be faked. Identity must come from your authentication layer rather than from user input.

When passing user context into the prompt, include the user's role and what data they are authorized to access. Only add extra context such as department, permission level, account identifiers, when it's needed to shape Claude's response. Don't include this information by default.

Data handling: what belongs in the context window

The context window is not a data-governance boundary. Any data passed into a Claude call is transmitted to the API. Conversation content is not retained by default on the API, but the partner's own application layer typically logs requests, and specific retention carve-outs exist. The architecture must make a deliberate decision about which fields need to be in the context window, and which should stay in the retrieval layer until needed.

For each field that enters the context window, ask whether it is necessary for Claude to produce the intended output. Reference identifiers like account numbers or claim numbers are often needed for routing but not for the language task itself. When a reference identifier is enough, passing the full data field needlessly exposes it to any request logging the partner's application layer performs, without adding any capability.

Data residency requirements vary by industry and region. For regulated deployments, verify Anthropic's data residency guidance against the specific regulatory requirements of your deployment before the integration is designed.

Observability: what to log, what to trace, and why

An LLM-based system is harder to debug than a traditional system because it doesn't crash when something goes wrong, instead it just produces a subtly wrong response. Standard logging catches errors and timeouts. It doesn't, however, catch a response that is quietly incorrect in a way that has real business consequences.

A production Claude system should log four things:

• The request: model version, input token count, prompt identifier
• The response: output token count, latency, stop reason
• The context: user role, session ID, whether caching was applied
• The outcome: whether the downstream system accepted the output and any rejection signals

Security organizations increasingly treat observability as the precondition for enabling agents at all, since without a trustworthy audit trail, an autonomous system is not approved to act. Design for that standard. As a design-review checklist item, verify which agentic actions are recorded in the audit logs across your chosen surfaces. Coverage varies by surface, and an action that is taken but not logged is, to a security reviewer, an action that cannot be allowed.

The PII field that went straight into the prompt

A trace excerpt from a regulated-industry deployment

The trace below is a composite of a field pattern in a healthcare-adjacent deployment. The team built a patient intake summarization tool. The summarization worked correctly. The data handling did not.

The SSN and Insurance ID are in the user message and therefore travel with the API request, exposed to any application-layer request logging, despite not being needed to produce the summary. The system prompt asks for presenting concerns, medications, and allergies. None of those require the patient's SSN or Insurance ID to be in the context window.

What broke

The tool worked as designed, but the data handling was wrong. Several fields that qualify as Protected Health Information (PHI) under HIPAA, the patient's name, date of birth, SSN, Insurance ID, chief complaint, medications, and allergies, were passed into the API call and captured in plaintext in the application layer's request logs. These were not necessary for the language task. When the deployment was reviewed prior to production certification, the application layer's request logs contained thousands of entries with patient SSNs in the user message field.

The fix was a data architecture change: a server-side redaction step that strips non-essential PII fields before the Claude call, and a retrieval function that supplies only the fields the language task needs. Both should have been in the original design.

Critique the integration diagram

The diagram below shows a Claude deployment for a customer-service agent at a multi-tenant SaaS company. Review the labeled components and connections, and select every one that represents an integration problem.

Select all problems. Leave sound components unselected.

A/B testing and observability at scale

Integration patterns get Claude into the enterprise stack. The question that follows is whether it is performing the way it should once it is there. Observability answers the monitoring question. Structured A/B testing answers the improvement question. Without both, you are either flying blind or making changes you cannot measure.

Structured A/B testing for live Claude systems

An A/B test for a Claude system follows the same structure as any experiment: a hypothesis, a treatment group, a control group, a metric, and a sample size large enough to make the result statistically meaningful. The difference from traditional software A/B testing is that LLM outputs are probabilistic, which makes the results noisier and the interaction effects harder to control.

The hypothesis must be specific and testable. "The new prompt is better" fails on both counts: it names no treatment, no metric, and no threshold. A usable hypothesis reads like this: "Replacing the instruction to summarize with an instruction to extract the three most important action items will increase task success rate by at least 5% without degrading response latency p95." It names the treatment, the metric, the threshold for success, and the constraint.

Reading results without overclaiming

Statistical significance means the result is unlikely to have occurred by chance given the sample size. Whether it is large enough to matter is a separate question. A change can be statistically significant but still too small to justify the operational cost of shipping and maintaining the new version.

The two questions to ask before declaring a winner are: is the effect large enough to justify the operational overhead of maintaining the new version? And did any secondary metric degrade? A prompt change that improves task success rate while increasing cost by 30% may not be a net win, depending on the budget constraints of the deployment.

LLM experiments have an additional failure mode that classical A/B tests do not: interaction effects between the treatment and specific input types. A prompt change that improves performance on typical inputs may degrade performance on edge-case inputs that appear rarely in the test period but frequently in a future seasonal spike. Test period and input distribution alignment matters more in LLM experiments than in most other software contexts.

Shadow testing: validating a change before any user sees it

A live A/B test sends real users to the new version, which means a regression reaches some fraction of them before the experiment closes. There is a way to test against real traffic without taking that exposure. You run the new version in parallel with the current one, send it a copy of live requests, and serve every user the current version's response. The new version's outputs are logged rather than returned, and you score them offline after the fact. The deployment decision is made before a single user has seen the new version. That pattern is called shadow testing.

The choice between the two patterns comes down to how much risk the deployment can carry and how much traffic it sees.

• Use a live A/B test when the deployment can absorb a small, bounded amount of exposure to a worse version and the traffic volume is high enough to reach a statistically meaningful sample in a reasonable window. The payoff is that you measure the new version against real user behavior, including the downstream signals a live response produces, such as whether the user accepted the answer or followed up.
• Use shadow testing when a single bad output carries too much risk, or when traffic is too low to support a live split before the change is needed. The cost of shadow testing is the loss of downstream signal: with no users receiving the shadow output, scoring relies on an offline rubric or golden answers rather than real user behavior. For a regulated-industry deployment, where exposing users to an unvalidated model change may not be permissible at all, shadow testing is often the only acceptable way to validate the change.

Observability at scale: instrumentation design, dashboards, anomaly detection

Production observability for a Claude system needs to answer four questions: what is the system doing, how well is it performing, when did it change, and why did it change? Each question requires a different layer of instrumentation.

• Request-level tracing. Every request should produce a trace that includes the model, model version, input token count, output token count, latency, stop reason, and any tool calls made. This is the raw material for everything else.
• Metric aggregation. Aggregate the request-level data into the metrics the dashboard displays: cost per request, latency p50 and p95, task success rate (if the downstream system provides an acceptance signal), and error rate by error type. Per-request decomposition is critical: aggregate metrics can look healthy while a small fraction of requests consume most of the budget.
• Anomaly detection. Set threshold alerts on the metrics that matter for the deployment. A cost spike that exceeds 150% of the 7-day average deserves an alert. A latency p95 that crosses the SLA threshold deserves an alert. Model drift (gradual change in the distribution of model outputs over time) is harder to detect with threshold alerts and benefits from periodic distribution comparison.
• Change attribution. When a metric moves, the instrumentation should be able to distinguish Model drift (the model's behavior on stable inputs changed), data drift (the input distribution changed), and model update effects (the model version changed and the new version behaves differently on existing inputs). These three causes have different mitigations and mixing them up produces the wrong fix.

A failure taxonomy: classifying what you are looking at

Instrumentation tells you a metric moved; diagnosis tells you what kind of failure moved it. Before attributing a change, classify the failure. The common classes are distinct and call for different fixes:

• Prompt failure. The instruction was ambiguous or underspecified and the model filled the gap. The fix is in the prompt, not the model.
• Hallucination. The model produced confident, fluent content that is not grounded in the input or a reliable source. The fix is grounding through retrieval, tool use, or verification. Stronger instruction will not resolve it.
• Model mismatch. The chosen tier is wrong for the task or was swapped without re-evaluation. The fix is model selection, gated by an eval.
• Orchestrator-workers failure. In multi-agent systems, trace across the orchestrator and its subagents: a recoverable subagent failure (retry or flag) looks different from an unrecoverable orchestrator failure. Attribution requires a trace that spans both.

Discernment: judging the output

Discernment is one of the four AI Fluency competencies, defined as the discipline of judging the quality of what the model produced rather than accepting it at face value. Applied to a production system, Discernment is the habit of classifying each output as acceptable, needs revision, or needs override, and feeding that judgment back into the evals and the monitoring. A team without Discernment watches metrics move and never asks whether the underlying outputs were actually good.

Connecting observability data to business value

The people who funded the Claude deployment are not reading the request-level trace. They are reading a KPI dashboard that measures the outcome the deployment was designed to improve. The observability stack needs a translation layer that connects the technical metrics to the business metrics they drive.

For a customer service agent, the business metric might be average handle time, first-contact resolution rate, or customer satisfaction score. The observability stack measures latency, task success rate, and error rate. The translation layer maps task success rate to first-contact resolution and latency to handle time, so the business owner can see whether the system is moving the numbers they care about.

Build this translation layer when the system is designed, not after the first business review. If the technical and business metrics are not mapped at build time, the first business review will raise a question about what is driving the change in handle time. Without that mapping, answering it requires a retrospective reconstruction rather than running a live query.

The 50-session winner that wasn't

The prompt change that looked like a win

The following is a composite representing a pattern that surfaces in teams that have a working system and want to improve it but do not have a formal experimentation process.

A team running a customer service agent wanted to test a revised instruction in the system prompt. They ran the new version against 50 customer sessions and the old version against 50 sessions. The task success rate was 68% on the new version and 62% on the old version. They declared the new version a winner and deployed it.

Two weeks later, the task success rate on the new version had settled at 61%. The apparent 6-point gain had disappeared.

What broke

The comparison had three problems, any of which was sufficient to invalidate the result.

• The sample size was too small. A 6-point difference on a metric with high variance requires a sample size in the hundreds to reach statistical significance. At 50 sessions per group, the observed difference was within the noise floor.
• The input distribution was not controlled. The 50 sessions in the treatment group happened to contain fewer edge-case inputs than the 50 sessions in the control group. The apparent improvement was partly an artifact of which inputs were routed to which group.
• The primary metric was not pre-specified. The team compared the task success rate because it moved in the right direction. If it had moved in the wrong direction, they would have looked at another metric. Selecting the metric after seeing the result turns a test into a search for whatever metric happened to move.

Place on the experiment-design plane

The plane below has two axes: the expected effect size (how large a difference you expect to see) and the confidence requirement (how certain you need to be before acting on the result). Place each scenario in the correct zone. The correct placement determines the right experimental posture, which in turn determines the minimum sample size.

Exercise: define the evaluation framework

Draft the evaluation framework for this system. For each of the five dimensions below, write the metric, the grading method (code-based eval, LLM judge, or human review), and a one-sentence reason for your choice. Write your framework, then reveal the model answer below.

[Click-to-reveal an answer to compare with your response; feel free to ask Claude to compare what you've written with the provided answer.]

Production readiness builder

Click-to-reveal answers to compare with your responses; feel free to ask Claude to compare what you’ve written with the provided answer.

Glossary

The key terms used across this module, in alphabetical order. Click a term to expand its definition.

Five takeaways

Key takeaways:

Module 3 covers responsible AI, safety, and risk for Architects: guardrail design, regulated-industry considerations, and human-in-the-loop validation strategies.

Sources

• Anthropic Skilljar, Building with the Claude API: eval workflow stages, model-based vs. code-based evals, cost and latency modeling, caching, tool use, streaming.
• Anthropic Skilljar, Claude 101: model family overview, context windows, general Claude capabilities.
• Anthropic Skilljar, Claude Code in Action: Claude Code as an integration entry point, agentic patterns in practice.
• Anthropic Skilljar, AI Capabilities and Limitations: four AI properties framework, foundational concepts.
• platform.claude.com/docs: models overview page (capability and context-window figures), pricing page (per-token price points for cost modeling), and prompt caching page (caching mechanics and consistency risk guidance).
• Anthropic, Building Effective Agents: workflow and agent design patterns; when to use agents vs. simpler architectures.
• Anthropic Cookbook: failure-mode candidates and worked patterns.

Congratulations! You have successfully completed this module.

Module 2 covers the integration patterns, production infrastructure, and operational decisions that take a Claude deployment from prototype to enterprise scale.

Production reliability is an architecture decision, not an operational one; you now have the patterns to make it at design time.