Case Study · Agentic AI
Multi-Agent Customer Support Platform
A Bedrock supervisor agent routes to specialist agents, calls tools, and retrieves from a knowledge base — with guardrails and human escalation built in from the start.
Bedrock AgentsOpenSearchSQSLambdaDynamoDB
01
The problem & requirements
Functional
- User submits a query via chat → system routes to the right specialist agent (order status, refunds, FAQ) and returns a coherent answer
- Agents call tools/APIs and retrieve from a knowledge base (RAG) for policy/FAQ questions
- Low-confidence or sensitive requests escalate to a human agent
Non-functional
- Latency: p95 under ~3s for a single-agent turn, up to ~8s for multi-hop tool use
- Reliability: tool calls must be idempotent — a retried refund shouldn't double-refund
- Safety: guardrails against prompt injection, PII leakage, off-policy responses
- Observability: full trace of which agent handled a request and why
- Cost control: token spend needs a model-tiering strategy, not one large model for everything
02
Scale & constraints
Assume 500 concurrent conversations, each turn averaging 1,500 input tokens and 300 output tokens.
~90,000/sec
sustained token throughput at peak — 1,800 tokens/turn × 500 sessions turning over every ~10s
2 calls
LLM invocations per turn — one small/fast routing call, one larger specialist call
RAG on path
retrieval latency sits before the specialist model responds — often the bigger lever than model choice
03
API design
POST /chat {session_id, message}
WebSocket for streaming responses.
Orchestrator → Bedrock Agent (routing)
Returns {agent: order_status | refund | faq, confidence}.
Agent → Action Group (Lambda)
Structured tool-call interface — Bedrock Agents define action groups via OpenAPI schema, e.g. `getOrderStatus(order_id)`
Agent → Knowledge Base . retrieve(query) → [{chunk, score, source}]
Against a Bedrock Knowledge Base backed by OpenSearch Serverless.
04
Data model
| Entity | Key fields |
|---|---|
| Session (DynamoDB) | session_id, user_id, conversation_history, active_agent, last_updated |
| Agent trace | trace_id, session_id, agent_invoked, tools_called[], latency_ms, confidence_score |
| Knowledge base doc | doc_id, source_url, chunk_text, embedding_vector, last_indexed |
| Tool call log | call_id, tool_name, params, idempotency_key, result, status |
idempotency_key on tool calls is what separates a working design from a naive one — without it, a retried Lambda invocation could double-process a refund.
05
Architecture
Client hits API Gateway, which authenticates and routes to a Bedrock orchestrator agent. The orchestrator hands off to specialist agents — order status and knowledge/FAQ — which call action-group Lambdas and the knowledge base directly.
👤
Client
Chat / voice
→
◈
API Gateway
Auth + routing
→
🧠
Orchestrator
Bedrock supervisor agent
↓
Specialist agents · Bedrock Agents runtime
📦
Order status agent
Calls backend APIs
📚
Knowledge / FAQ agent
RAG retrieval
↓
Tools & data
ƒ
Action groups
Lambda tool calls
▲
Knowledge base
OpenSearch vectors
DynamoDB session stateCloudWatch / X-Ray tracingBedrock GuardrailsSQS escalation
Fig. 2 — Orchestrator routes to specialist agents, which call tools and the knowledge base in parallel.
06
Key decisions & trade-offs
Bottleneck — sequential agent hops. Orchestrator → specialist → tool call → synthesis can be 3–4 round trips. Use a fast/cheap routing model and let independent tool calls run in parallel, synthesizing once both return.
Bottleneck — RAG retrieval on the critical path. Cache frequent queries, pre-fetch likely-relevant docs from the routing classification, keep chunk size small.
Trade-off — single agent vs. multi-agent orchestration. Multi-agent buys modularity and per-task accuracy at the cost of coordination complexity and added latency per hop.
Reliability. A refund Lambda must dedupe on
idempotency_key since agent retries could otherwise trigger the same side effect twice.Safety. Bedrock Guardrails filter input and output; low-confidence routing or guardrail trips push the session to SQS for human pickup rather than letting the agent guess.
System
Multi-Agent Customer Support Platform
Primary services
Bedrock Agents · OpenSearch · SQS
Status
Architecture complete
Type
Multi-agent orchestration