<p>Modern Deep Learning technologies such as Self Attention and RNN, kind of integrated higher-degree nonlinear functions <strong>fundamentally</strong>. e.g. <span class="math-container">$(w_{1_{branch1}}x_1 + w_{2_{branch1}}x_2) \cdot (w_{1_{branch2}}x_1 + w_{2_{branch2}}x_2)$</span> .</p>
<p>Besides, there aren't too much development on e.g. trigonometric function AFAIK. From Bionics perspective, is there any nonlinear-functional neuron actually exists in human brain? If so, what kind of exact function do they encode?</p>
<p>Related question:</p>
<p><a href="https://psychology.stackexchange.com/q/21315">Is there a biological equivalent to the bias term used in artificial neural networks?</a></p>
<p><a href="https://psychology.stackexchange.com/q/13379">What are the rules that govern neuron behavior?</a></p>

CC BY-SA 4.0
2024-02-11T15:28:52.033Z
All biological neurons are nonlinear.
Bryan Krause
CC BY-SA 4.0
2024-02-11T16:23:12.777Z
@BryanKrause Are they all unable to be ideally treated as linear? If so, what is the similarity between Artificial Neural Network and "real neural network"? Apologize for my limited understanding about biology.
LifeJustDLC
CC BY-SA 4.0
2024-02-11T16:46:24.723Z
Not very similar, just loosely motivated. If you want to learn about neurobiology, I'd recommend starting from the basics and learning bottom-up, rather than starting from what you know about ANN/ML and working backwards making assumptions. In the "all models are wrong but some are useful" sense we might sometimes treat biological neurons as linear in order to more easily simulate populations to test particular assumptions, but it's sort of a "take a uniform spherical cow" situation in physics where everyone knows the model is wrong.
Bryan Krause
CC BY-SA 4.0
2024-02-11T17:45:23.740Z
@BryanKrause Didn't know "spherical cow", thanks for your informative replay though. These well working models maybe far from "correct".
LifeJustDLC
CC BY-SA 4.0
2024-02-11T18:26:35.813Z
There are at least two categories of neuron "model": those that are useful for learning about the brain and those that are useful for solving other tasks. They're pretty much mutually exclusive, because the things that are mechanistically easy with biology are computationally really expensive, and vice versa. Artificial networks tend to become less and less brain-like the more they are optimized to solve tasks (I.e. machine learning/AI)
Bryan Krause
<neuroscience><neural-network><artificial-intelligence>
Is there any type of neuron capable of modeling "nonlinear function" (e.g. trigonometric, instead of "linear" in traditional Neural Networks)?
LifeJustDLC
# Is there any type of neuron capable of modeling "nonlinear function" (e.g. trigonometric, instead of "linear" in traditional Neural Networks)?

(Asked by: username_0 on 2024-02-11)

Modern Deep Learning technologies such as Self Attention and RNN, kind of integrated higher-degree nonlinear functions **fundamentally**. e.g. $(w\_{1\_{branch1}}x\_1 + w\_{2\_{branch1}}x\_2) \cdot (w\_{1\_{branch2}}x\_1 + w\_{2\_{branch2}}x\_2)$ .

Besides, there aren't too much development on e.g. trigonometric function AFAIK. From Bionics perspective, is there any nonlinear-functional neuron actually exists in human brain? If so, what kind of exact function do they encode?

Related question:

[Is there a biological equivalent to the bias term used in artificial neural networks?](https://psychology.stackexchange.com/q/21315)

[What are the rules that govern neuron behavior?](https://psychology.stackexchange.com/q/13379)

**Tags:** neuroscience, neural-network, artificial-intelligence

> **Comments:**
> **username_1**: All biological neurons are nonlinear.  
> **username_0**: @BryanKrause Are they all unable to be ideally treated as linear? If so, what is the similarity between Artificial Neural Network and "real neural network"? Apologize for my limited understanding about biology.  
> **username_1**: Not very similar, just loosely motivated. If you want to learn about neurobiology, I'd recommend starting from the basics and learning bottom-up, rather than starting from what you know about ANN/ML and working backwards making assumptions. In the "all models are wrong but some are useful" sense we might sometimes treat biological neurons as linear in order to more easily simulate populations to test particular assumptions, but it's sort of a "take a uniform spherical cow" situation in physics where everyone knows the model is wrong.  
> **username_0**: @BryanKrause Didn't know "spherical cow", thanks for your informative replay though. These well working models maybe far from "correct".  
> **username_1**: There are at least two categories of neuron "model": those that are useful for learning about the brain and those that are useful for solving other tasks. They're pretty much mutually exclusive, because the things that are mechanistically easy with biology are computationally really expensive, and vice versa. Artificial networks tend to become less and less brain-like the more they are optimized to solve tasks (I.e. machine learning/AI)  

---

## 0 answers

