Amazon Bedrock: A Practitioner's Read
What Bedrock gets right, what it still doesn't, and how we use it in production alongside Claude and OpenAI. Notes from shipping four systems on the platform.
Bedrock is AWS’s answer to the question every enterprise asks six months into an AI initiative: “how do we keep our data inside our perimeter while still getting access to frontier models?” The answer is tolerably good and genuinely useful, with caveats that matter.
We’ve shipped four production systems on Bedrock over the past year. Here is the honest read.
What Bedrock gets right
IAM and VPC integration. For teams that are already AWS-native, Bedrock’s access control story is its real product. Keeping inference traffic inside a VPC, auditing every invocation via CloudTrail, and gating access by IAM role is the kind of boring infrastructure that makes security teams approve your proposal instead of rejecting it. Nothing else competes on this axis.
Knowledge Bases with vector store options. The managed RAG flow — chunking, embedding, syncing from S3 — works out of the box and integrates with OpenSearch Serverless, Aurora PostgreSQL with pgvector, and Pinecone. It is not better than a hand-rolled pipeline. It is better than not having a pipeline at all, which is where most teams actually start.
Model choice. Claude, Llama, Mistral, Titan, Cohere — all available through a single API with consistent auth. For teams that want to route by task (Claude for reasoning, Llama for cost-sensitive bulk work, Titan for multimodal), this is genuinely useful.
What Bedrock still doesn’t do well
Cold start latency. First-token latency on Bedrock consistently lags direct API calls by 200-400ms. For conversational UX, this is perceptible. For batch workloads, it is irrelevant. Know which you are shipping.
Model feature parity. Bedrock tends to lag upstream providers by weeks to months on new capabilities. Prompt caching, the latest function-calling schemas, and some advanced features land on Anthropic’s and OpenAI’s direct APIs well before they reach Bedrock. If you need the feature today, go direct.
Observability. CloudWatch metrics for Bedrock are thin. For real production observability — token accounting, prompt analysis, cost attribution per user — you will build your own layer on top. Budget for that work.
Bedrock is the right answer for teams who need AWS-native compliance. It is the wrong answer for teams chasing the frontier.
How we actually use it
Our default pattern: Bedrock for anything touching regulated data (healthcare, financial, legal), direct Anthropic and OpenAI APIs for everything else. The routing lives in a thin abstraction layer that normalizes request/response shapes so we can swap providers without rewriting application code.
- Clarity SI (med-legal): Bedrock + Claude, VPC-contained, with CloudTrail audit logging for HIPAA compliance.
- Concierge Health: Bedrock for PHI-adjacent workloads, direct Anthropic for non-sensitive assistance features.
- Terrein (audit engine): Direct APIs — no compliance constraint, latency matters.
- StatStaq (sports analytics): Mixed — Bedrock for the infrastructure playbook, direct calls for the user-facing real-time features.
The decision tree
Use Bedrock if: you have an AWS-first security posture, you need VPC isolation, your compliance team requires single-tenant inference, or you want model optionality without juggling vendor contracts.
Skip Bedrock if: you need the latest model capabilities the day they ship, you are optimizing for sub-second first-token latency, or you’re small enough that the direct APIs with SOC 2 already satisfy your security review.
Most serious production systems end up with both. Treat Bedrock as one tool in the routing layer, not as the routing layer itself.