153. LLM Inference with Bedrock

<p>If you’re curious about building with LLMs, but you want to skip the hype and learn what it takes to ship something reliable in production, this episode is for you.We share our real-world experience building AI-powered apps and the gotchas you hit after the demo: tokens and cost, quotas and throttling, IAM and access friction, marketplace subscriptions, and structured outputs that do not break your JSON parser.We focus on Amazon Bedrock as AWS’s managed inference layer: how to get started with the current access model, how to choose models, how pricing works, and what to watch for in production.We also go deep on structured outputs: constrained decoding, schema design that improves output quality, and how to avoid “grammar compilation timed out”.</p><p><br></p><p>In this episode, we mentioned the following resources:</p><ul><li>fourTheorem: Bedrock structured outputs guide <a href="https://fourtheorem.com/amazon-bedrock-structured-outputs/" target="_blank" rel="noopener noreferer">https://fourtheorem.com/amazon-bedrock-structured-outputs/</a></li><li>Amazon Bedrock <a href="https://aws.amazon.com/bedrock/" target="_blank" rel="noopener noreferer">https://aws.amazon.com/bedrock/</a></li><li>Bedrock docs <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/" target="_blank" rel="noopener noreferer">https://docs.aws.amazon.com/bedrock/latest/userguide/</a></li><li>Bedrock pricing <a href="https://aws.amazon.com/bedrock/pricing/" target="_blank" rel="noopener noreferer">https://aws.amazon.com/bedrock/pricing/</a></li><li>Structured outputs <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/structured-outputs.html" target="_blank" rel="noopener noreferer">https://docs.aws.amazon.com/bedrock/latest/userguide/structured-outputs.html</a></li><li>Cross-region inference <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html" target="_blank" rel="noopener noreferer">https://docs.aws.amazon.com/bedrock/latest/userguide/cross-region-inference.html</a></li><li>Quotas <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html" target="_blank" rel="noopener noreferer">https://docs.aws.amazon.com/bedrock/latest/userguide/quotas.html</a></li><li>Throttling help <a href="https://repost.aws/knowledge-center/bedrock-throttling-error" target="_blank" rel="noopener noreferer">https://repost.aws/knowledge-center/bedrock-throttling-error</a></li><li>Prompt caching <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html" target="_blank" rel="noopener noreferer">https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html</a></li><li>Troubleshooting error codes <a href="https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html" target="_blank" rel="noopener noreferer">https://docs.aws.amazon.com/bedrock/latest/userguide/troubleshooting-api-error-codes.html</a></li></ul><p><br></p><p>Do you have any AWS questions you would like us to address?</p><p>Leave a comment here or connect with us on X/Twitter, BlueSky or LinkedIn:</p><p><br></p><p>- <a href="https://twitter.com/eoins">⁠https://twitter.com/eoins⁠</a> | <a href="https://bsky.app/profile/eoin.sh">⁠https://bsky.app/profile/eoin.sh⁠</a> | <a href="https://www.linkedin.com/in/eoins/">⁠https://www.linkedin.com/in/eoins/⁠</a></p><p>- <a href="https://twitter.com/loige">⁠https://twitter.com/loige⁠</a> | <a href="https://bsky.app/profile/loige.co">⁠https://bsky.app/profile/loige.co⁠</a> | <a href="https://www.linkedin.com/in/lucianomammino/">⁠https://www.linkedin.com/in/lucianomammino/</a></p>