TY - JOUR T1 - Embedding Inequalities for Barron-Type Spaces AU - Wu , Lei JO - Journal of Machine Learning VL - 4 SP - 259 EP - 270 PY - 2023 DA - 2023/12 SN - 2 DO - http://doi.org/10.4208/jml.230530 UR - https://global-sci.org/intro/article_detail/jml/22307.html KW - Barron space, Two-layer neural network, High-dimensional approximation, Embedding theorem, Fourier transform. AB -

An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω),$ where the index $s ∈ [0, ∞)$ indicates the smoothness of functions within these spaces and $Ω ⊂ \mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: For any $\delta∈ (0, 1),s ∈ \mathbb{N}^+$ and $f : Ω\mapsto \mathbb{R},$ it holds that $$\delta||f||_{\mathcal{F}_{s−\delta}(Ω)}\lesssim_s||f||_{\mathcal{B}_s(Ω)}\lesssim_s||f||_{\mathcal{F}_{s+1}(Ω).}$$ Importantly, the constants do not depend on the input dimension $d,$ suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.