arXiv · Oct 15, 2026
Optimal approximation of smooth functions by ReLU neural networks
By Thomas Nagler
We study uniform approximation of the H"older ball mathcal H₁^s([0,1]^D) by fully connected ReLU networks. Except in the one-dimensional Lipschitz case, existing constructions remain separated from the information-theoretic efficiency limit. This article closes the gap for every s>0 and D≥1: networks with width O(N), depth O(L), and parameter magnitude O(1) attain uniform approximation error of order (N²L²log N)^(-s/D). No relation between width and depth is required. The construction uses new encoding and decoding mechanisms that exploit higher-order finite-difference bounds for local Taylor coefficients of the target function. As a statistical consequence, empirical risk minimizers over the corresponding clipped network classes attain the minimax regression rate n^(-2s/(2s+D)) without additional logarithmic factors. The paper also gives a detailed exposition of the key approximation techniques, covering both established and new mechanisms.