When you build an application on top of a large language model (LLM), the prompt you send to the model typically has two parts. There’s a fixed part that sets up context (instructions, reference documents, conversation history) and a variable part that contains the actual user input. Take a customer service bot as an example. […]
Reduce LLM latency with prefix-aware routing on Amazon SageMaker Inference Read More »
When you deploy a large language model (LLM) for inference on Amazon SageMaker HyperPod, there’s a gap between when you request a pod and when it’s ready to serve traffic. This gap is dominated by two sequential downloads: the inference server container image from Amazon Elastic Container Registry (Amazon ECR), and the model weights from
Reduce inference cold starts on Amazon SageMaker HyperPod with model caching Read More »
Today we’re announcing the general availability of TwelveLabs Marengo Embed 3.0 as an embedding model in Amazon Bedrock Knowledge Bases. Video and media assets remain largely unsearchable by meaning. Teams in media, sports analytics, education, security, and retail need to find specific moments in hours of footage using natural language. An example query is “show
Video and image search in Amazon Bedrock Knowledge Base using Marengo 3.0 Read More »
Your teams get an AI assistant that handles real work while your data stays in your environment and your conversations stay private Today, the Amazon Quick desktop application is generally available on macOS and Windows. We’re also adding a new activity feed to the mobile experience on iOS and Android that consolidates email, calendar, CRM,
Amazon Quick is now generally available on desktop Read More »
Build an end-to-end Request for Information (RFI) questionnaire workflow using Amazon Quick Automate to solve a challenge organizations face at every scale. A typical enterprise might handle hundreds of RFI questionnaires each year, and each arrives as a complex, multi-tab workbook packed with hierarchical question sets, category metadata, and varied response types. The volume, variety,
Build an end-to-end RFI questionnaire workflow using Amazon Quick Automate Read More »
A configurable, instruction-driven detector that runs on any large language model (LLM) managed on Amazon Bedrock, evaluated on five public PII corpora across nine LLM-based detectors, including the OpenAI PrivacyFilter. Fine-tuning a model on real-world text creates a personally identifiable information (PII) detection problem. Training corpora are full of PII: names, home addresses, email and
Model-agnostic PII detection with LLMs Read More »
Search can help runners get race-day ready with registration alerts, tailored training plans, and more.
3 ways to prep for your next big race with Search Read More »
Multi-turn agents fail in ways that single-turn evaluation misses: one early mistake quietly corrupts every later turn. This post introduces the Agent Evaluation Metric (AEM), a decomposable, turn-level way to measure agent quality. We apply it to its first dimension, correctness. We show how AEM pinpoints the one turn that caused a failure and separates
Agent Evaluation Metric for multi-turn conversations Read More »
This post is co-written with Petra Lafond, Product Manager, and Maarten Cardinaels, Tech Lead at AvioBook, a Thales Group Company. Airlines run on tight, cascading schedules, and accessing operational data holds the key to protecting them. During turnarounds, a few minutes of delay at one gate can ripple through an entire day of connections. It
On August 12, 2026, Alibaba’s Qwen team released Qwen3.8-2.4T-A95B. This is the first time a Qwen-Max-class model has been made available as open weights. With 2.4 trillion total parameters (95 billion activated per token), a hybrid linear-plus-full-attention architecture, and native context up to 262K tokens (extensible to 1M), Qwen3.8 targets the most demanding agentic and
Deploying Qwen3.8-2.4T-A95B on Amazon SageMaker HyperPod with vLLM Read More »
