J. Mach. Learn. , 2 (2023), pp. 259-270.

Published online: 2023-12

[*An open-access
article; the PDF
is free to any online user.*]

Cited by

- BibTex
- RIS
- TXT

An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω),$ where the index $s ∈ [0, ∞)$ indicates the smoothness of functions within these spaces and $Ω ⊂ \mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: For any $\delta∈ (0, 1),s ∈ \mathbb{N}^+$ and $f : Ω\mapsto \mathbb{R},$ it holds that $$\delta||f||_{\mathcal{F}_{s−\delta}(Ω)}\lesssim_s||f||_{\mathcal{B}_s(Ω)}\lesssim_s||f||_{\mathcal{F}_{s+1}(Ω).}$$ Importantly, the constants do not depend on the input dimension $d,$ suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.

}, issn = {2790-2048}, doi = {https://doi.org/10.4208/jml.230530}, url = {http://global-sci.org/intro/article_detail/jml/22307.html} }An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}_s(Ω)$ and the spectral Barron space $\mathcal{F}_s(Ω),$ where the index $s ∈ [0, ∞)$ indicates the smoothness of functions within these spaces and $Ω ⊂ \mathbb{R}^d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: For any $\delta∈ (0, 1),s ∈ \mathbb{N}^+$ and $f : Ω\mapsto \mathbb{R},$ it holds that $$\delta||f||_{\mathcal{F}_{s−\delta}(Ω)}\lesssim_s||f||_{\mathcal{B}_s(Ω)}\lesssim_s||f||_{\mathcal{F}_{s+1}(Ω).}$$ Importantly, the constants do not depend on the input dimension $d,$ suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.

*Journal of Machine Learning*.

*2*(4). 259-270. doi:10.4208/jml.230530