Amazon SageMaker now supports Multi-Instance GPU to cut AI inference costs

Deploying AI models to GPUs has traditionally forced a rigid choice: either pay for a whole accelerator that a smaller model cannot fully utilize, or struggle with complex self-managed Kubernetes stacks. With native GPU partitioning on SageMaker AI, users can now slice accelerators like the A100 or H100 into independent partitions with hardware-enforced isolation. AWS tests show that running a model on a partitioned newer GPU can actually be faster and cheaper than using a whole older GPU. For example, a partitioned H100 slice outperformed a whole A100 while significantly lowering the cost per million tokens. This capability makes it much easier to run a portfolio of models on the same hardware without performance interference.