AWS Neuron 2.32 introduces expanded NKI programming, MXFP8 training kernels, and variable-size collectives for Trn2 and Trn

AWS Neuron 2.32.0 is now available, featuring NKI 0.6.0 with on-device top-K instructions, variable-length all-gather for differently sized tensors, and new loop constructs for data-dependent iteration. The update adds 13 new NKI Library kernels for MoE training and sparse attention, plus variable-size collectives for Trn2 and Trn3. The vLLM Neuron plugin has been updated to vLLM 0.24.0 and is included in all Neuron Deep Learning AMIs and containers.

Why it matters

This update impacts users of Trn2 and Trn3 instances, especially those training large transformer and MoE models