arXiv · Oct 20, 2026
Two ReLU Layers is All You Need
By Florestan Brunck, Christoph Hertrich, Kilian Ruess et al.
We prove that every ReLU network can be exactly simulated by a ReLU network with two hidden layers. We present two proofs: one that is based on fillings in homological algebra and a direct combinatorial proof.