Optimizing cost per task and proving agent ROI on Amazon Bedrock

Part 1 of this series set up the operating model that keeps Amazon Bedrock spending under control: attribution through AWS Identity and Access Management (IAM) principal tags and Cost and Usage Report 2.0, observability through model invocation logging, enforcement at three tiers, and a regular review cadence. That layer identifies who spent the money and lets you stop them, but it cannot say whether the spend was worthwhile.

This second installment answers the two questions that were deliberately left open. First, why is a given workflow so expensive? Answering that means reasoning in cost per task rather than cost per token. Second, is the agent actually better than the humans were? That calls for baselines, controlled experiments, and a scorecard tied to decisions.

The framework below covers how to compute a model's true cost, including four hidden cost drivers that never appear on the pricing page; how to cut cost per task with caching, routing, and output constraints; and how to measure agent ROI using human baselines, holdout experiments, and a five-dimension scorecard. It is aimed at two audiences: the application and platform engineers who own the optimization levers, and the product owners and FinOps partners who must defend agent spending to leadership.

Cost per task, not cost per token

Price per token is convenient for quick comparisons, but it does not describe what you actually pay. The published rate is the price of a single token; your bill is the price of finishing a task. A task is one complete unit of work your application asks of a model — resolving one support email, extracting fields from one invoice, or answering one user question — measured from the request through to a usable result, however many model calls that takes.

The lower-priced model is not automatically the cheaper one to run. A nominally cheaper model may split your text into more tokens, pad its answers with filler, misread your tool schema and loop through corrections, or spend thousands of reasoning tokens before producing an answer. None of that shows up on the Bedrock pricing page.

The four hidden cost drivers

Four model characteristics determine what a published rate really costs on your workload.

Tokenizer efficiency

A token is not a word, and tokens are not equivalent across models. Every model family ships a different tokenizer and vocabulary, so the same text, code, or JSON payload breaks into different token counts. If Model A's list price sits 15 percent lower but its tokenizer needs 40 percent more tokens on your data — a common situation with structured schemas and numerical arrays — then Model B is cheaper. The gap widens for non-English text: many tokenizers fragment non-Latin scripts into two to three times more tokens than the equivalent English.

Output alignment and verbosity

Output tokens commonly cost three to five times more than input tokens, though the ratio varies by family (refer to the Bedrock pricing page). A model that ignores formatting constraints such as "return only raw JSON" makes you pay that output premium on filler words. A model that follows formatting instructions closely generates fewer of these more expensive output tokens and therefore saves money.

Agentic loop efficiency

Agentic workflows change the cost math entirely. Tool use (function calling) lets a model invoke external functions in a multi-turn loop: analyze the prompt, decide to call a tool, wait for the observation, then generate an answer. Every turn resends the full conversation history and tool results as input tokens. A model that needs four turns, each resending a larger context payload, costs far more than one that finishes in a single pass.

Reasoning overhead

Many models support extended thinking, working through a chain of thought before the visible response. Reasoning tokens are billed as output tokens, so a model that thinks for 2,000 tokens before a 200-token answer charges you for 2,200. Some models redact the reasoning while still billing for it. Leave reasoning turned off by default and enable it only when benchmarks reveal a quality gap that justifies the cost.

A worked example

Consider an agentic workflow that reads a 750-word customer email, extracts entities, calls an API to verify account status, and returns strict JSON. The comparison below uses illustrative prices — always benchmark against your own data and current Bedrock pricing.

Model A: (6,150 x $0.50 + 750 x $1.50) / 1,000,000 = $0.0042

Model B: (1,000 x $1.00 + 150 x $3.00) / 1,000,000 = $0.00145

The model priced twice as high ends up costing 65 percent less because of three compounding effects: Model A's tokenizer fragments the email into 50 percent more tokens, inflating every input payload; it pads its JSON with filler, tripling output tokens; and it misaligns with the tool schema, requiring two correction turns that each resend a growing history.

Cost per successful task

That example assumes both models eventually succeed. In production they will not, and a failure is rarely free. For retry-until-success workflows:

cost per successful task = cost per attempt / success rate

For escalate-on-failure workflows:

cost per successful task = cost per attempt + (1 - success rate) x escalation cost

With prompt caching active, input further splits into three buckets: uncached input billed at the base rate, cache reads billed below it, and cache writes billed above it.

Measuring it on your own workload

Which model is genuinely cheaper only emerges when you benchmark against your own data. Amazon Bedrock model evaluation is the managed feature covering the quality half of that work: you define criteria, upload your prompt datasets, and get scored results from automatic metrics with optional human review, with no custom infrastructure required.

Quality scores alone do not produce a bill, so three more Bedrock features supply the token and turn counts to price against them:

• CountTokens reports the input token count a given model would charge for a given payload without running inference. It is the cheapest way to compare tokenizer efficiency (driver one) across candidates before spending anything on generation.
• The model invocation response carries a usage block with inputTokens, outputTokens, cacheReadInputTokens, and cacheWriteInputTokens. Accumulating them across the turns of a single task provides the four terms of the cost-per-attempt formula above. One caveat: when caching is enabled, inputTokens counts only the uncached tokens, so total input volume equals inputTokens + cacheReadInputTokens + cacheWriteInputTokens. Sum them to know the footprint, but price them separately, because the three bill at different rates. This evaluation should be repeated throughout the application's life and cache checkpoints should track the current active load: as the application progresses and tools go stale, checkpoints drift, which means more cache writes three months later. Constantly evaluating cache usage becomes a primary lever for reducing cost.
• Model invocation logging writes per-invocation token counts to Amazon Simple Storage Service (Amazon S3) or Amazon CloudWatch Logs, letting you measure turns-to-completion against real production traffic instead of a sample. Tagging calls with request metadata lets you slice logged token counts by team, feature, or experiment. These are log fields, however, not cost allocation tags, and they never appear in AWS Cost Explorer or your Cost and Usage Report. Application inference profiles do reach both, but at a grain of one usage type per day: right for chargeback, too coarse for cost per task. Multiplying logged token counts by published rates yields a near real-time estimate rather than an invoice, and it will not reflect commitments, batch pricing, or provisioned throughput unless you model them yourself. Reconcile it against your Cost and Usage Report before treating it as the bill.

Reducing cost per task: four levers

Each driver has a matching lever, and each is documented in depth. Prompt caching removes the resent context that made Model A expensive, making it the largest win on agentic paths. Intelligent prompt routing sends easy requests to a cheaper model in the same family. A tool schema combined with a maxTokens cap holds verbosity down more reliably than prose instructions, and extended thinking stays off until a benchmark shows it earns back its cost. Batch inference halves the rate on bulk single-turn work, though it rules out tool calling and multi-turn exchange.

Proving agent ROI: describe the human baseline

Efficient is not the same as valuable. A perfectly optimized agent can still not be worth running, and the question that decides budgets is whether the agent is better than the humans were.

Document the human process first

The biggest mistake in agent ROI is trying to prove value without first describing the human workflow. Before building or scaling an agent, write down the current process in operational terms: what triggers the work, which human roles are involved, where the handoffs sit, and how completion is defined. It may sound like bureaucracy, but it is actually measurement design, because every downstream instrumentation decision — what counts as a run, what counts as success, where human time is spent — falls out of this description.

Measure the baseline before the agent ships

Once the process is described, measure the human baseline over a meaningful window: time to completion, queue time, active effort, reopen rate, escalation rate, cost per task, and outcome quality. Baselines must be captured before the agent goes live, not reconstructed from memory afterward. Retrospective baselines systematically flatter the agent: people remember the average case and forget the fast ones.

Instrument the workflow as an experiment

With a baseline in place, instrument the agent workflow as an experiment from day one. Every run gets a trace ID, and every meaningful step is observable. A workflow-run record aggregates everything above the model call into a single analyzable unit:

{
  "runId": "run-8f3a2c",
  "workflow": "renewal-risk-summary",
  "agent": "renewal-agent",
  "principalArn": "arn:aws:sts::123456789012:assumed-role/BedrockRuntimeRole/gw-alex",
  "team": "sales-ai",
  "costCenter": "CC-2041",
  "experiment": {
    "variant": "treatment",
    "holdout": False,
    "promptVersion": "v14",
    "policyVersion": "2026-07"
  },
  "ops": {
    "latencyMs": 41200,
    "retries": 1,
    "toolCalls": 6,
    "humanApprovals": 1,
    "humanMinutes": 35,
    "escalations": 0
  },
  "econ": {
    "inputTokens": 48210,
    "outputTokens": 3105,
    "cacheReadTokens": 31500,
    "estCostUsd": 0.87
  },
  "outcome": {
    "status": "success",
    "accepted": True,
    "reopened": False,
    "revenueEventId": "opp-44121"
  }
}

Three fields deserve emphasis:

• experiment block: stamped when the run is created and propagated into Bedrock request metadata, so model-level and workflow-level records join cleanly.
• humanMinutes: the field most teams omit and most need. Without it, "the agent did the work" silently excludes the 35 minutes a human spent reviewing it.
• revenueEventId: the hook that makes revenue attribution possible later. It costs nothing to record and cannot be reconstructed afterward.

The rule: if the workflow telemetry cannot answer the business question, the instrumentation is incomplete.

Join cost to outcomes

With invocation logs landing in Amazon S3 (per Part 1) and workflow-run records in your analytics store, a single join produces the metrics that matter. Bedrock logs record tokens rather than dollars, so first build a view that prices each invocation from token counts and your model rate sheet, aggregated by run ID, then query it with Amazon Athena:

SELECT
  w.workflow,
  w.variant,
  COUNT(*) AS runs,
  SUM(IF(w.outcome_status = 'success', 1, 0)) AS successes,
  ROUND(SUM(c.est_cost_usd) /
    NULLIF(SUM(IF(w.outcome_status = 'success', 1, 0)), 0), 2)
    AS cost_per_success,
  ROUND(SUM(w.retries) * 1.0 /
    NULLIF(SUM(IF(w.outcome_status = 'success', 1, 0)), 0), 2)
    AS retries_per_success,
  ROUND(SUM(w.human_minutes) /
    NULLIF(SUM(IF(w.outcome_status = 'success', 1, 0)), 0), 1)
    AS human_min_per_success
FROM workflow_runs w
JOIN invocation_costs_by_run c ON c.run_id = w.run_id
GROUP BY w.workflow, w.variant
ORDER BY cost_per_success DESC

The column names and join keys in that query are illustrative; verify them against your invocation log schema and workflow telemetry store, which can vary by implementation.

Cost per successful outcome, retries per success, and human minutes per success are the three numbers that survive contact with a skeptical CFO. Cost per call flatters agents that loop; cost per success does not. Note the variant grouping: the same query split by experiment arm is the ROI comparison.

Compare agents and humans correctly

The most credible evaluation is an A/B or holdout test, where controlled treatment and control groups compare the agent-assisted workflow against the human baseline without confusing the result with market, seasonal, or organizational changes.

For customer-facing or revenue-linked workflows, randomize at the visitor, lead, account, or case level. For internal workflows such as support operations or coding assistants, use matched cohorts: teams, repository groups, or pods.

The comparison covers four dimensions:

• Efficiency: time to completion, hours recovered, queue-time reduction.
• Quality: error rate, approval rate, reopen rate, escalation rate.
• Cost: cost per task, cost per successful outcome, total cost to serve.
• Business value: conversion lift, retention lift, expansion rate, revenue per rep or per workflow.

If the agent improves one dimension while meaningfully degrading another, the result is ambiguous and the deployment needs tuning.

Revenue: the hardest metric

Efficiency gains are the easiest part of the ROI story to prove. Revenue impact is harder, and it is the metric leadership funds against. In revenue-adjacent workflows, measure whether the agent improves conversion, time-to-response, deal progression, or revenue per rep. Holdout tests are the most reliable way to separate incremental revenue attributable to the agent from general business momentum.

When revenue lift is real, the conversation shifts: the organization stops asking whether the Bedrock bill went up and starts asking whether the additional spend buys a meaningful increase in business output.

The scorecard

Based on what the authors have observed working with customers, a strong operating model does not require fifty metrics. What works is a small scorecard tied to decisions, reviewed on a regular cadence for every production agent. The thresholds below are starting points, not prescriptions — calibrate them against your own fleet data and organizational context.

Green means scale: the agent beats baseline while meeting cost and quality thresholds. Yellow means tune: promising, but it needs work. Red means retire or redesign: the agent consumes cost without enough measurable value, and the willingness to actually retire a red agent is what lends the rest of the scorecard credibility.

Conclusion

Calculating a model's true cost means accounting for the four hidden drivers — tokenizer efficiency, output alignment, agentic loop efficiency, and reasoning overhead. Reducing cost per task runs through prompt caching, model routing, output constraints, and batch inference. Proving agent ROI means measuring human baselines, instrumenting workflows as experiments, joining cost to outcomes, and running holdout comparisons.

The cost-optimization half explains why a workflow is expensive and how to fix it; the ROI half answers whether the agent is better than the humans were. Together with Part 1's operating model — attribute, observe, enforce, optimize, operate — they form a complete framework: you can see who spent the money, explain why, reduce the waste, and justify what remains.

To get started, benchmark your top model candidates against your own prompts with Amazon Bedrock model evaluation, capture tokens and turns from the usage block or your invocation logs, price them against current Bedrock pricing, and divide by success rate. For ROI measurement, consult the Well-Architected Agentic AI Lens for agent cost governance best practices. The authors also invite readers to share in the comments any other techniques they have found for reducing cost per task or measuring agent ROI on Amazon Bedrock.