<p>After having read matus's beautiful answer in <a href="https://cstheory.stackexchange.com/questions/17545/universal-approximation-theorem-neural-networks/17630#17630?newreg=bb92f636fd6443e9bc21a23c5924808b">this thread</a> explaining (among other things) Kolmogorov's result regarding the Universal Approximation Theorem with Neural Networks, I wonder: if just $\mathcal{O}$($d^2$) nodes can replicate (!) the target function, provided we can use different transfer functions, do there exist theory or algorithms for inferring from the training data which transfer functions to use?</p>

CC BY-SA 3.0
2014-01-20T15:37:09.263Z
my understanding, the applied theory has gone more in the direction of identifying some standard transfer functions with parameter adjustments that [nearly] support universality. ie mainly the sigmoid function...
vzn
CC BY-SA 3.0
2014-01-21T00:25:46.840Z
So it seems, but why? I crave for theory that justifies the path taken by applied research! Deep learning shouldn't be a black box...
Alex Dalyac
<approximation-algorithms><machine-learning><ne.neural-evol><st.statistics>
Practical Implications of Kolmogorov's Result on the Universal Approximation Theorem with Neural Networks
Alex Dalyac
# Practical Implications of Kolmogorov's Result on the Universal Approximation Theorem with Neural Networks

(Asked by: username_0 on 2014-01-20)

After having read matus's beautiful answer in [this thread](https://cstheory.stackexchange.com/questions/17545/universal-approximation-theorem-neural-networks/17630#17630?newreg=bb92f636fd6443e9bc21a23c5924808b) explaining (among other things) Kolmogorov's result regarding the Universal Approximation Theorem with Neural Networks, I wonder: if just $\mathcal{O}$($d^2$) nodes can replicate (!) the target function, provided we can use different transfer functions, do there exist theory or algorithms for inferring from the training data which transfer functions to use?

**Tags:** approximation-algorithms, machine-learning, ne.neural-evol, st.statistics
> **Comments:**
> **username_1**: my understanding, the applied theory has gone more in the direction of identifying some standard transfer functions with parameter adjustments that [nearly] support universality. ie mainly the sigmoid function...  
> **username_0**: So it seems, but why? I crave for theory that justifies the path taken by applied research! Deep learning shouldn't be a black box...  

---

## 0 answers

