arXiv · Oct 19, 2026
Solving the Ho–Nguyen Conjecture for Gaussian Mixtures: Applications to Mixtures of Experts
By Yuyang Zhang, Qihuang Zhang
Over-fitted Gaussian mixtures can contain many parameter configurations with nearly the same density, making the order of density cancellation central to their parameter-learning rates. Ho and Nguyen encoded these cancellations in a polynomial system and conjectured its exact order as a function of cluster size. The key to resolving the conjecture is a common positive-variance shift that turns the coefficient equations into Gaussian moment matching. For every cluster size Mhn≥2, we prove that the real, nonzero-weight system has order 2Mhn: a sharp sign-change bound forces rigidity at that order, while positive Gauss–Hermite quadrature constructs a non-trivial solution through order 2Mhn-1. Population-density calculations at 110-digit working precision recover the predicted leading term for two to six components; finite multistart searches recover the same observed powers for two to five components, with smaller finite-scale constants for three to five components. For parameter learning, the exact order gives an explicit pointwise maximum-likelihood exponent and an exact local minimax order on specified shrinking neighbourhoods of a fixed known mixing measure. The same coefficient viewpoint separates high-order cancellation from exact parameter symmetries in mixtures of experts. Synchronized replication preserves the conditional density while keeping the corresponding parameter losses positive in two broad standard-softmax classes, showing why statistical rate arguments must account for exact parameter symmetries.