Amazon SageMaker AI Adds Usage-Based Pricing for Real-Time Streaming

According to AWS Machine Learning:

Background

Real-time speech AI applications like voice agents and live transcription rely on bidirectional streaming. However, billing for these long-lived sessions was difficult because traditional AWS Marketplace software metering only worked for standard request-response APIs, where the billable quantity is known instantly at the end of a request. For live streams lasting 30 minutes, hourly pricing often charged customers even during quiet periods, failing to reflect true consumption.

Technical Details

To fix this, Amazon SageMaker AI now opens a second WebSocket connection alongside the main data stream. This companion metadata connection operates on a dedicated path, allowing model servers to send JSON text frames reporting incremental usage. Providers send metrics using the inference.count dimension and ConsumedUnits quantities. Developers can choose whether a dropped metadata connection should terminate the inference session or continue in data-only mode, balancing reliability with metering continuity.

Industry Impact

This update fundamentally aligns software charges with actual consumption for real-time AI workloads. Enterprise customers benefit from consolidated AWS billing and private offers without managing separate vendor relationships, while model providers gain a standardized path to monetize streaming AI seamlessly through AWS Marketplace.