An AWS Machine Learning Blog post published on 09.11.2026 presents the Pluggable Agentic AI Framework, a composable reference architecture organized into seven layers and designed to take multi-agent AI applications from prototype to production scale on Amazon Bedrock AgentCore.
The post opens with the questions a proof of concept cannot answer: how to handle thousands of concurrent user requests securely, how to keep agents reliable under real traffic instead of failing silently, how to know whether an agent still meets the quality and safety standards it passed during testing, and which agent behavior should be deterministic or non-deterministic. The authors argue that the gap between a working prototype and a system that runs reliably, securely, frugally, and responsibly at scale is where most agentic AI initiatives stall, and that it is rarely the model's fault. Teams tend to rebuild the same scaffolding, including routing logic, guardrails, and observability, applied after the fact rather than designed in from the start.
The seven layers
Each layer has a distinct responsibility and a stable contract. The services named in each layer reflect the reference implementation rather than mandatory choices; organizations can substitute alternatives that align with their infrastructure and requirements.
Data and Knowledge. This foundation layer owns retrieval, embeddings, and access to structured and unstructured data through a contract that accepts a query and returns the most relevant information as context for the agent. Example building blocks include Amazon Bedrock Knowledge Base, Vector Engine for Amazon OpenSearch Serverless, Amazon S3 Vectors, and Amazon DynamoDB.
Model and Inference. Model choice should be driven by configuration rather than hardwired into agent logic: smaller, faster models for simple tasks and larger models for requests requiring more nuanced reasoning, selected after evaluation on realistic and representative business data. A model routing layer keeps agents decoupled from the hosting environment and provider-specific APIs; a proxy such as LiteLLM provides a common interface across providers so models can be switched without rewriting application logic.
Orchestration. For straightforward tasks the authors recommend a single agent end to end; for complex cross-domain workflows, a supervisor that delegates to specialist agents, each scoped to a specific capability and tool set. They warn against one agent that does everything: accumulating tools, instructions, and context grows the context window, increases latency, can reduce accuracy, and produces tool sprawl that can push agents beyond their intended boundaries toward unintended actions and security breaches. Specialists should operate under bounded autonomy, executing strictly within granted tools. The post also recommends decoupling specialists from the supervisor through a centralized agent registry that catalogs each agent's tools, skills, and owner, with invocation through standardized protocols such as MCP (Model Context Protocol) and A2A (Agent to Agent). Suggested components include Strands Agents for specialists, the Amazon Bedrock AgentCore harness for the orchestrator, and AWS Agent Registry with Amazon Bedrock AgentCore Gateway for centralized discovery and routing.
Security and Guardrails. The authors state that agentic AI exposes organizations to risks conventional security frameworks were not built to address, from prompt injection and tool misuse to memory and data poisoning. Recommendations include treating all external content as untrusted, authenticating every request whether it comes from a user, another agent, or an automated workflow (for example through Amazon Bedrock AgentCore Identity), applying fine-grained access control before each tool call, granting temporary scoped credentials instead of persistent secrets, and masking sensitive data such as PHI and PII in requests and responses. For accountability, the post calls for a tamper-evident audit trail of significant decisions, a human approval gate for high-impact actions, and circuit breakers to halt anomalous behavior. Suggested services include Amazon Bedrock Guardrails for user inputs, AgentCore Identity and Policy in Amazon Bedrock AgentCore for authentication and access control, and AWS security and governance controls for defense in depth.
Application. The end-user experience layer, whether a chat interface, API surface, or embedded widget, should stay intentionally thin: it presents the conversation and passes requests to the orchestration layer, without embedded business logic. A decision model can sit at this boundary as a fast, inexpensive check that scores whether a request is in scope, needs a human, or is urgent, returning probabilities rather than free text so that the full agentic loop runs only when it adds value. The post also advises designing external interfaces so that other AI agents, not only humans, can consume and interact with the system.
Observability. Monitoring should cover every layer from day one. Alongside logs and metrics, end-to-end agent traces are critical for measuring whether the agent performed its task correctly; the post also lists cost and token usage per request, time to first token, whether each call is secure, and dashboards for latency, error rates, and spend before going live. Suggested tooling includes Amazon CloudWatch with OpenTelemetry and Amazon CloudWatch Omni for tracing and metrics.
Agentic Operations. This layer closes the loop with continuous evaluation: scoring every production session for groundedness, accuracy, and safety, with a regression gate to catch quality drops before they reach customers. Suggested components are Amazon Bedrock AgentCore Evaluations for scoring and Amazon Bedrock AgentCore optimization for continuous performance tuning.
Reference implementation
The framework ships with a ready-to-run reference implementation built on a retail customer support scenario covering order status and refund eligibility, chosen because it exercises every layer while remaining easy to adapt. A supervisor agent receives a refund request and delegates to an Order agent and a Refund agent. The Order agent looks up delivery and purchase details in Amazon DynamoDB; the Refund agent retrieves tier-based policy rules from a knowledge base and calculates eligibility using the delivery date, the current date, and the customer's loyalty tier. Every call passes through AgentCore Gateway, which enforces fine-grained access control before a tool executes via Policy in AgentCore, and every call is traced end to end so a faulty calculation can be attributed to a specific layer and decision point. Teams building a similar supervisor-and-specialist pattern can clone the implementation and swap in their own domain logic layer by layer.
Why pluggability matters
The authors argue that the value of the layered design lies in how quickly each layer can evolve: stable contracts at layer boundaries, from identity checks and access policies to telemetry interfaces, allow the underlying technology to change without redesigning the stack. The framework demonstrates this through integrations with third-party providers across layers, including model and hosting providers at inference, search capabilities at orchestration, decision APIs at the application layer, guardrails at the security layer, and agent tracing and monitoring at observability alongside business value and cost tracking. For enterprises, the post says, this flexibility protects existing investments and avoids requiring every technology decision to come from the same vendor.
The post concludes that a prototype proves what an agent can do while architecture determines whether an enterprise can depend on it, and recommends starting with the Workshop Studio experience and the public sample repository, with the caveat that the implementation is not a substitute for production security hardening, testing, and review.