NVIDIA Audex: 30B mixture-of-experts model unifies speech, audio, and text

NVIDIA released Audex, a mixture-of-experts model that unifies audio, speech, and text processing in a single architecture. The 30-billion-parameter system activates 3 billion parameters per forward pass and handles speech synthesis, transcription, translation, audio generation, and sound event detection while preserving text-based reasoning capabilities.

The model supports context windows up to one million tokens and ships in two sizes: the full 30B version and a compact 2B variant for edge deployment. At 2 billion parameters, the smaller model fits comfortably in 16GB of VRAM and can run on mid-range GPUs; the 30B version requires multi-GPU setups but delivers higher fidelity on complex audio tasks.

Audex avoids the performance trade-offs typical in multimodal systems. Most architectures either bolt audio encoders onto frozen language backbones—sacrificing audio fidelity—or train audio-native models that struggle with reasoning. The mixture-of-experts design routes different token types through specialized sub-networks while keeping the parameter budget manageable. The million-token context window is notable for an audio-language model; audio tokens are expensive, and a single minute of speech can consume thousands of tokens depending on encoding. A context budget that large lets Audex process multi-hour recordings or maintain state across extended conversational turns without truncation.