E-PMQ recovers 42 points on 20-task CLIP by stabilizing merged-model quantization

E-PMQ, a post-merge quantization framework from researchers across multiple institutions, addresses accuracy collapse when quantizing neural networks built by merging multiple task-specific experts. Posted to arXiv on May 19, 2026, the preprint shows that applying standard post-training quantization (PTQ) to a merged model produces unreliable results because quantization error and merging error compound.

The framework uses source expert weights as calibration targets and anchors merged weights during layer-wise reconstruction to preserve the integrated behavior of the merged model. On CLIP-ViT-B/32 merged across eight tasks, E-PMQ improved 4-bit GPTQ accuracy from 65.0% to 73.6% under Task Arithmetic merging and from 69.1% to 74.8% under TIES-Merging. Gains grow larger in harder settings: 20-task CLIP-ViT-L/14 jumped from 34.8% to 76.7%, and FLAN-T5-base on GLUE rose from 78.26% to 83.34%.

What stands out

1. 01Dual-deviation problem identified. The paper isolates two distinct error sources in post-merge quantization: the quantization deviation from low-bit reconstruction and the expert-relative merging deviation inherited from the merge itself. Standard PTQ couples these errors, causing accuracy to collapse.
2. 02Expert-guided calibration stabilizes reconstruction. E-PMQ feeds the original expert weights as output targets during layer-wise calibration, giving the quantizer a reference signal that reflects the pre-merge behavior of each expert and preventing drift.
3. 03Merged-weight anchoring preserves integrated structure. The framework anchors the merged weights during calibration to prevent the quantizer from drifting away from the merged model's intended behavior.
4. 04Consistent gains across merging methods. E-PMQ lifted 4-bit accuracy by 8.6 points under Task Arithmetic and 5.7 points under TIES-Merging on the eight-task CLIP-ViT-B/32 benchmark, demonstrating effectiveness across multiple merging strategies.