<p>Because of variation between organisms, cells in the same organism, or even the same cell separated by a few days, can have different parameter set. However, I think this is where parameter fitting becomes useful. As suggested in the comments knowing how parameters change between cells or over time can be very insightful. Parameter Estimation is very important because it allows us gather more information about the parameter set from less experimental data and thus allowing more complex experiments.  </p>

<p>Philosophy aside, here are some tips for getting into the field. I'd suggest reading this open access article by Van Giet <em>et al.</em> <a href="https://www.researchgate.net/profile/Erik_De_Schutter/publication/23476235_Automated_neuron_model_optimization_techniques_a_review/links/0912f50bc00097d685000000.pdf" rel="nofollow noreferrer">here</a> and I'll give a brief review of their article. </p>

<p><strong>General Model</strong>
First one needs a general framework to work within,</p>

<ul>
<li>What is your type of neuron model going to be?  a integrate and fire neuron(discontinuous reset) or is electrophysilogical model (continuous). </li>
<li>What are the currents/channels/gates in it (Sodium, Potassium, Chlorine, Calcium)?</li>
<li>what form do the differential equations take?</li>
<li>What are the known/unknown parameters?</li>
</ul>

<p>The  more  you can answer these questions the easier the parameter fitting process will be, but beware, a one size fits all approach rarely works in biology because of cell to cell variation.</p>

<p><strong>Error Function</strong>
Next one needs what is called an error function, or a way to tell if the model output voltage trace is similar to the actual voltage trace. The most classical (but in my honest opinion the worst one)  is the $\mathcal{L}_2$ norm on the voltages. The  $\mathcal{L}_2$ is simply </p>

<p>$$\sqrt{\sum^{N}_i (V_{model}(i)-V_{data}(i))^2}$$ </p>

<p>To see why this is so bad I attached a sample (simulated trace from the Hodgkin-Huxley model). </p>

<p><a href="https://i.sstatic.net/nqUPz.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/nqUPz.jpg" alt="Two Voltage Traces"></a></p>

<p>The Blue Trace is from a Neuron with slightly less injected current then the yellow Trace one. As you can see yellow has a faster frequency than blue, and as a result the  $\mathcal{L}_2$ norm is massive  for each misaligned spike. However the spike trains are intuitively very close as seen not only in where the spikes line up but where the spikes so one would like a measure that defines misaligned phase and frequency. </p>

<p>Van Geit <em>et al's.</em> insight is that the  $\mathcal{L}_2$ is the wrong norm to compare voltage distance in. What they do is make a parametric plot of voltage verses voltage's derivative. The then compare distance between the curves like in the below plot.</p>

<p><a href="https://i.sstatic.net/pfF0C.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/pfF0C.jpg" alt="enter image description here"></a>.</p>

<p>We see that the voltage traces are indeed very similar, but to calculate distance between these curves, we must disregard time data. We can imagine subdividing phase space into a grid of equally sized boxes. Then count up the number of data points and model points in the boxes. In math notation this amounts to</p>

<p>$$
\sqrt{\sum^{N_x,N_y}_{i,j} (V_{model}(i,j)-V_{data}(i,j))} 
$$  </p>

<p>we have lost information about phase of the spike. It is also important to note that this is <em>highly dependent</em> on the size of the boxes, if they are too small the error will be high, too big the error will be to low.  While there method may not be the only proposed solution it does better than the $\mathcal{L}_2$</p>

<p><a href="https://i.sstatic.net/Yrcvc.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/Yrcvc.jpg" alt="enter image description here"></a></p>

<p>Here is both methods side by side for comparison. The black dot is the true solution. Blue is $\mathcal{L}_2$ and Yellow is Van Geit's method. You'll note that both are ``noisy" but there is a better trend in Van Geit's method than the $\mathcal{L}_2$ norm, thus optimization is easier. For the $\mathcal{L}_2$ method notice how the function is flat far away from a very steep and narrow valley. This is hard for optimization algorithms to hit. Van Geit's method is better because it has more of a downward slope that can be followed to a better global minima.</p>

<p>Also  Note that this noise is not from random variation (although that makes it worse) but rather from discretization of the data and model. (remember everything stored in a computer has a sample rate no matter how small). </p>

<p><strong>Optimization Algorithm</strong>
Now Once has an appropriate error function one needs to use a optimization algorithm too find the local minimum of said error function. In my example here, I am using injected current as my parameter, but in every neuron model has many parameters that one needs to optimize over so direct visualization like is usually impossible. </p>

<p>Another caveat is the ``noise". these local valleys can make optimization algorithms get stuck in the local minima. Stochastic (random) optimization algorithms like simulated annealing  can help with getting stuck in these valleys, as they have a chance to jump past them. </p>

<p><strong>Good Initial guesses</strong>
As far as not responding the "full space'' of solutions, the more of the possible inputs one has the better chance of being able to predict a different input. Also defining the computational behavior of your neuron is a good start.  Having a general model that acts qualitatively correctly but not quantitatively is often a good start to parameter fitting, This means one doesn't start with a neuron that doesn't respond to inputs, or something else physically unrealistic. </p>

<p>Anyway hope this gives a background to things to consider, this field is far from solved, and in need of creative solutions.</p>

CC BY-SA 3.0
2016-03-31T13:27:18.730Z
This ended up way longer than I expected. Hope this helps and If I need to clarify anything let me know.
xelo747
CC BY-SA 3.0
2016-03-31T13:25:55.637Z
2016-03-31T13:25:55.637Z
xelo747
<p>I'm hoping to contribute to the OpenWorm project by helping their efforts to parameterize the neurons in CElegans so the model elicits biologically realistic behavior.</p>

<p>The problem is I've got only five activity series datasets for all neurons in CE and I fear whatever model is trained on these datasets won't be an accurate model of the full space of neural activity in CElegans. </p>

<p>Can you give some ideas as to how the field works around this and where I can read about what methods are used to resolve this issue?</p>

CC BY-SA 3.0
2016-03-29T16:10:07.703Z
you do what you can.  later people will refine it.  that is the best you can do.  all biophysically realistic computational models are woefully underconstrained at this point.
honi
CC BY-SA 3.0
2016-03-29T18:23:39.320Z
So what is the worth of your model then?
theideasmith
CC BY-SA 3.0
2016-03-30T15:11:20.993Z
well there are several benefits. one is that with a model you can explore the effects of changing unknown parameter values, potentially providing predictions for what those parameters should be.  another is that with a model, you can do experiments that you couldn't do in an actual animal because you actually have access to all of the relevant variables. it doesn't give you a slam dunk answer but it gives you another perspective on the problem.  the model can also be iteratively refined by making predictions, testing them experimentally, and then improving the model based on those experiments.
honi
<theoretical-neuroscience><computational-modeling>
Problems with parameterizing neuronal simulation on a limited number of datasets
theideasmith
# Problems with parameterizing neuronal simulation on a limited number of datasets

(Asked by: username_0 on 2016-03-29)

I'm hoping to contribute to the OpenWorm project by helping their efforts to parameterize the neurons in CElegans so the model elicits biologically realistic behavior.

The problem is I've got only five activity series datasets for all neurons in CE and I fear whatever model is trained on these datasets won't be an accurate model of the full space of neural activity in CElegans.

Can you give some ideas as to how the field works around this and where I can read about what methods are used to resolve this issue?

**Tags:** theoretical-neuroscience, computational-modeling

> **Comments:**
> **username_1**: you do what you can.  later people will refine it.  that is the best you can do.  all biophysically realistic computational models are woefully underconstrained at this point.  
> **username_0**: So what is the worth of your model then?  
> **username_1**: well there are several benefits. one is that with a model you can explore the effects of changing unknown parameter values, potentially providing predictions for what those parameters should be.  another is that with a model, you can do experiments that you couldn't do in an actual animal because you actually have access to all of the relevant variables. it doesn't give you a slam dunk answer but it gives you another perspective on the problem.  the model can also be iteratively refined by making predictions, testing them experimentally, and then improving the model based on those experiments.  

---

## 1 answers

### Answer by username_2 on 2016-03-31. Score: 2 | Accepted answer

Because of variation between organisms, cells in the same organism, or even the same cell separated by a few days, can have different parameter set. However, I think this is where parameter fitting becomes useful. As suggested in the comments knowing how parameters change between cells or over time can be very insightful. Parameter Estimation is very important because it allows us gather more information about the parameter set from less experimental data and thus allowing more complex experiments.

Philosophy aside, here are some tips for getting into the field. I'd suggest reading this open access article by Van Giet *et al.* [here](https://www.researchgate.net/profile/Erik_De_Schutter/publication/23476235_Automated_neuron_model_optimization_techniques_a_review/links/0912f50bc00097d685000000.pdf) and I'll give a brief review of their article.

**General Model**
First one needs a general framework to work within,

* What is your type of neuron model going to be? a integrate and fire neuron(discontinuous reset) or is electrophysilogical model (continuous).
* What are the currents/channels/gates in it (Sodium, Potassium, Chlorine, Calcium)?
* what form do the differential equations take?
* What are the known/unknown parameters?

The more you can answer these questions the easier the parameter fitting process will be, but beware, a one size fits all approach rarely works in biology because of cell to cell variation.

**Error Function**
Next one needs what is called an error function, or a way to tell if the model output voltage trace is similar to the actual voltage trace. The most classical (but in my honest opinion the worst one) is the $\mathcal{L}\_2$ norm on the voltages. The $\mathcal{L}\_2$ is simply

$$\sqrt{\sum^{N}\_i (V\_{model}(i)-V\_{data}(i))^2}$$

To see why this is so bad I attached a sample (simulated trace from the Hodgkin-Huxley model).

[![Two Voltage Traces](https://i.sstatic.net/nqUPz.jpg)](https://i.sstatic.net/nqUPz.jpg)

The Blue Trace is from a Neuron with slightly less injected current then the yellow Trace one. As you can see yellow has a faster frequency than blue, and as a result the $\mathcal{L}\_2$ norm is massive for each misaligned spike. However the spike trains are intuitively very close as seen not only in where the spikes line up but where the spikes so one would like a measure that defines misaligned phase and frequency.

Van Geit *et al's.* insight is that the $\mathcal{L}\_2$ is the wrong norm to compare voltage distance in. What they do is make a parametric plot of voltage verses voltage's derivative. The then compare distance between the curves like in the below plot.

[![enter image description here](https://i.sstatic.net/pfF0C.jpg)](https://i.sstatic.net/pfF0C.jpg).

We see that the voltage traces are indeed very similar, but to calculate distance between these curves, we must disregard time data. We can imagine subdividing phase space into a grid of equally sized boxes. Then count up the number of data points and model points in the boxes. In math notation this amounts to

$$
\sqrt{\sum^{N\_x,N\_y}\_{i,j} (V\_{model}(i,j)-V\_{data}(i,j))}
$$

we have lost information about phase of the spike. It is also important to note that this is *highly dependent* on the size of the boxes, if they are too small the error will be high, too big the error will be to low. While there method may not be the only proposed solution it does better than the $\mathcal{L}\_2$

[![enter image description here](https://i.sstatic.net/Yrcvc.jpg)](https://i.sstatic.net/Yrcvc.jpg)

Here is both methods side by side for comparison. The black dot is the true solution. Blue is $\mathcal{L}\_2$ and Yellow is Van Geit's method. You'll note that both are ``noisy" but there is a better trend in Van Geit's method than the $\mathcal{L}\_2$ norm, thus optimization is easier. For the $\mathcal{L}\_2$ method notice how the function is flat far away from a very steep and narrow valley. This is hard for optimization algorithms to hit. Van Geit's method is better because it has more of a downward slope that can be followed to a better global minima.

Also Note that this noise is not from random variation (although that makes it worse) but rather from discretization of the data and model. (remember everything stored in a computer has a sample rate no matter how small).

**Optimization Algorithm**
Now Once has an appropriate error function one needs to use a optimization algorithm too find the local minimum of said error function. In my example here, I am using injected current as my parameter, but in every neuron model has many parameters that one needs to optimize over so direct visualization like is usually impossible.

Another caveat is the ``noise". these local valleys can make optimization algorithms get stuck in the local minima. Stochastic (random) optimization algorithms like simulated annealing can help with getting stuck in these valleys, as they have a chance to jump past them.

**Good Initial guesses**
As far as not responding the "full space'' of solutions, the more of the possible inputs one has the better chance of being able to predict a different input. Also defining the computational behavior of your neuron is a good start. Having a general model that acts qualitatively correctly but not quantitatively is often a good start to parameter fitting, This means one doesn't start with a neuron that doesn't respond to inputs, or something else physically unrealistic.

Anyway hope this gives a background to things to consider, this field is far from solved, and in need of creative solutions.

> **Comments:**
> **username_2**: This ended up way longer than I expected. Hope this helps and If I need to clarify anything let me know.  

---

