NVFP4

NVFP4 is a 4-bit floating-point number format (“NV” = NVIDIA, “FP4” = 4-bit float). It squeezes a model’s weights to roughly a quarter of the size of BF16.

The benefit is big: lower memory use and a noticeably faster prefill, especially at short and medium context. As context grows, the advantage shrinks.

Important nuance from our test: NVFP4 files are not interchangeable. The one tuned for a specific model and engine (“Radics”) was fastest and kept quality; a third-party quant of the same model did not.