Blog_dumb

Monitoring production agent lifecycle with AWS DevOps Agent and AgentCore Evaluations

Monitoring production agent lifecycle with AWS DevOps Agent and AgentCore Evaluations

Multi-agent systems in production experience issues in ways that traditional monitoring misses. For example, the agent can’t invoke its foundation model (FM) and returns an empty response. This could be because of a missing AWS Identity and Access Management (IAM) permission on an agent’s execution role that doesn’t throw a 500 error. A supervisor agent […]

Monitoring production agent lifecycle with AWS DevOps Agent and AgentCore Evaluations Read More »

Beyond the price per token: Choosing the right OpenAI model on Amazon Bedrock for your workload

Beyond the price per token: Choosing the right OpenAI model on Amazon Bedrock for your workload

Organizations building generative AI applications usually compare models the same way: dollars per million tokens. It’s the number on every pricing page, so it becomes the number in every spreadsheet. But production workloads don’t buy tokens. They buy outcomes: a resolved support ticket, a completed research brief, a correct financial summary. Between the pricing page

Beyond the price per token: Choosing the right OpenAI model on Amazon Bedrock for your workload Read More »

Build interactive MCP Apps using Amazon Bedrock AgentCore

Build interactive MCP Apps using Amazon Bedrock AgentCore

As customers shift to interacting with digital services through AI hosts like ChatGPT and Claude, organizations need a way to make their services accessible across these applications with rich UI, not only plain text. They also need to do this without coupling to a single host. MCP Apps and Amazon Bedrock AgentCore provide exactly that.

Build interactive MCP Apps using Amazon Bedrock AgentCore Read More »

Reduce LLM latency with prefix-aware routing on Amazon SageMaker Inference

Reduce LLM latency with prefix-aware routing on Amazon SageMaker Inference

When you build an application on top of a large language model (LLM), the prompt you send to the model typically has two parts. There’s a fixed part that sets up context (instructions, reference documents, conversation history) and a variable part that contains the actual user input. Take a customer service bot as an example.

Reduce LLM latency with prefix-aware routing on Amazon SageMaker Inference Read More »

Reduce inference cold starts on Amazon SageMaker HyperPod with model caching

Reduce inference cold starts on Amazon SageMaker HyperPod with model caching

When you deploy a large language model (LLM) for inference on Amazon SageMaker HyperPod, there’s a gap between when you request a pod and when it’s ready to serve traffic. This gap is dominated by two sequential downloads: the inference server container image from Amazon Elastic Container Registry (Amazon ECR), and the model weights from

Reduce inference cold starts on Amazon SageMaker HyperPod with model caching Read More »

Video and image search in Amazon Bedrock Knowledge Base using Marengo 3.0

Video and image search in Amazon Bedrock Knowledge Base using Marengo 3.0

Today we’re announcing the general availability of TwelveLabs Marengo Embed 3.0 as an embedding model in Amazon Bedrock Knowledge Bases. Video and media assets remain largely unsearchable by meaning. Teams in media, sports analytics, education, security, and retail need to find specific moments in hours of footage using natural language. An example query is “show

Video and image search in Amazon Bedrock Knowledge Base using Marengo 3.0 Read More »

Build an end-to-end RFI questionnaire workflow using Amazon Quick Automate

Build an end-to-end RFI questionnaire workflow using Amazon Quick Automate

Build an end-to-end Request for Information (RFI) questionnaire workflow using Amazon Quick Automate to solve a challenge organizations face at every scale. A typical enterprise might handle hundreds of RFI questionnaires each year, and each arrives as a complex, multi-tab workbook packed with hierarchical question sets, category metadata, and varied response types. The volume, variety,

Build an end-to-end RFI questionnaire workflow using Amazon Quick Automate Read More »

Model-agnostic PII detection with LLMs

Model-agnostic PII detection with LLMs

A configurable, instruction-driven detector that runs on any large language model (LLM) managed on Amazon Bedrock, evaluated on five public PII corpora across nine LLM-based detectors, including the OpenAI PrivacyFilter. Fine-tuning a model on real-world text creates a personally identifiable information (PII) detection problem. Training corpora are full of PII: names, home addresses, email and

Model-agnostic PII detection with LLMs Read More »

Scroll to Top