<p>If your function is <span class="math-container">$f:\mathbb{R}\to\mathbb{R}$</span>, you can "learn" <span class="math-container">$f'$</span> as a standard regression problem (linear, polynomial, etc.) and then recover <span class="math-container">$f$</span> up to an additive constant by integrating <span class="math-container">$f'$</span>. Obviously, you will only able to recover <span class="math-container">$f$</span> up to an additive constant from the derivative.</p>

<p>Update: As for error estimates, here's a simple one. Suppose you've managed to recover <span class="math-container">$f'$</span> up to <span class="math-container">$\epsilon$</span>-accuracy in <span class="math-container">$L_\infty$</span>. Suppose further that you know that <span class="math-container">$f(0)=0$</span> and you only care about <span class="math-container">$f$</span> on <span class="math-container">$[0,T]$</span>. Then your estimate of <span class="math-container">$f$</span> (obtained by integrating <span class="math-container">$f'$</span>) is accurate up to <span class="math-container">$T\epsilon$</span> in <span class="math-container">$L_\infty$</span>.</p>

CC BY-SA 3.0
2016-12-05T03:13:21.440Z
Thank you for your kind example. However, as you said, when $f$ is estimated from $f'$ with error $\epsilon$, the error of estimation linearly grows (i.e. $T\epsilon$). In particular, if the interval of integration goes to infinity such as $[-\infty,\infty] = \mathbb{R} $, then the cumulative error $\int\epsilon dt$ also goes to infinity. So the error of estimation through the whole input domain always goes to infinity.In this case, is it possible to discuss the same thing as your answer? Minor question aoubt notation, does $L_{\infty}$ mean the set of function whose infinite norm is bounded?
Kyoungjae Lee
CC BY-SA 3.0
2016-12-05T08:00:37.860Z
If you assume something about the noise, you might be able to argue that the errors in the derivative cancel out on average (plausible, since you're integrating) and the deviation is bounded over all of $R$.
Aryeh
CC BY-SA 4.0
2019-09-03T20:17:36.693Z
Note that you might want to use a different loss function here than in a standard setting as error "accumulates" when you integrate. Particularly, points near the base point of integration matter more than points far from it. And you don't want to do catastrophically badly on a small set of inputs, whereas in a normal setting that's acceptable.
Reinstate Monica
CC BY-SA 4.0
2016-12-04T15:10:37.023Z
2019-09-01T14:16:40.460Z
2019-09-01T14:16:40.460Z
Aryeh
<p>In many machine learning algorithm, it is often assumed that outputs of unknown function and their corresponding inputs are given to estimate the unknown function. However, I wonder whether there exist some algorithms which can estimate unknown function using derivative data of unknown function and their corresponding inputs. For example,</p>

<p><span class="math-container">$$
\mbox{Given }D = {(x_1, y_1),\ldots,(x_n, y_n)}$$</span>
<span class="math-container">$$\mbox{ where }y_{1} = \frac{d{f(x)}}{dx}\Bigg|_{x=x_1},\ldots,y_n=\frac{d{f(x)}}{dx}\Bigg|_{x=x_n}$$</span>
<span class="math-container">$$\mbox{ Find  } f(x).$$</span></p>

<p>I found one curve fitting method which is called Hermite interpolation in <a href="https://en.wikipedia.org/wiki/Hermite_interpolation" rel="nofollow noreferrer">wiki</a> and some papers about <a href="http://ac.els-cdn.com/S0377042709008061/1-s2.0-S0377042709008061-main.pdf?_tid=db24720a-ba22-11e6-9fe0-00000aab0f27&amp;acdnat=1480857135_f6e56bec5913546a268d022d9a7dfc8e" rel="nofollow noreferrer">Hermite learning</a>. However, in these methods, outputs of unknown function and derivative outputs of it and corresponding inputs are given. Especailly, I want to know there exist some methods that utilize only derivative data. Also is it possible to find unknown function from derivative data, in particular, the error bound between unknown function and our estimator can decrease as the number of training data increases. How can we prove it?</p>

<machine-learning><lg.learning><st.statistics><pac-learning>
Learning from derivative data
Kyoungjae Lee
# Learning from derivative data

(Asked by: username_0 on 2016-12-04)

In many machine learning algorithm, it is often assumed that outputs of unknown function and their corresponding inputs are given to estimate the unknown function. However, I wonder whether there exist some algorithms which can estimate unknown function using derivative data of unknown function and their corresponding inputs. For example,

$$
\mbox{Given }D = {(x\_1, y\_1),\ldots,(x\_n, y\_n)}$$
$$\mbox{ where }y\_{1} = \frac{d{f(x)}}{dx}\Bigg|\_{x=x\_1},\ldots,y\_n=\frac{d{f(x)}}{dx}\Bigg|\_{x=x\_n}$$
$$\mbox{ Find } f(x).$$

I found one curve fitting method which is called Hermite interpolation in [wiki](https://en.wikipedia.org/wiki/Hermite_interpolation) and some papers about [Hermite learning](http://ac.els-cdn.com/S0377042709008061/1-s2.0-S0377042709008061-main.pdf?_tid=db24720a-ba22-11e6-9fe0-00000aab0f27&acdnat=1480857135_f6e56bec5913546a268d022d9a7dfc8e). However, in these methods, outputs of unknown function and derivative outputs of it and corresponding inputs are given. Especailly, I want to know there exist some methods that utilize only derivative data. Also is it possible to find unknown function from derivative data, in particular, the error bound between unknown function and our estimator can decrease as the number of training data increases. How can we prove it?

**Tags:** machine-learning, lg.learning, st.statistics, pac-learning

---

## 1 answers

### Answer by username_1 on 2016-12-04. Score: 4 | Accepted answer

If your function is $f:\mathbb{R}\to\mathbb{R}$, you can "learn" $f'$ as a standard regression problem (linear, polynomial, etc.) and then recover $f$ up to an additive constant by integrating $f'$. Obviously, you will only able to recover $f$ up to an additive constant from the derivative.

Update: As for error estimates, here's a simple one. Suppose you've managed to recover $f'$ up to $\epsilon$-accuracy in $L\_\infty$. Suppose further that you know that $f(0)=0$ and you only care about $f$ on $[0,T]$. Then your estimate of $f$ (obtained by integrating $f'$) is accurate up to $T\epsilon$ in $L\_\infty$.

> **Comments:**
> **username_0**: Thank you for your kind example. However, as you said, when $f$ is estimated from $f'$ with error $\epsilon$, the error of estimation linearly grows (i.e. $T\epsilon$). In particular, if the interval of integration goes to infinity such as $[-\infty,\infty] = \mathbb{R} $, then the cumulative error $\int\epsilon dt$ also goes to infinity. So the error of estimation through the whole input domain always goes to infinity.In this case, is it possible to discuss the same thing as your answer? Minor question aoubt notation, does $L_{\infty}$ mean the set of function whose infinite norm is bounded?  
> **username_1**: If you assume something about the noise, you might be able to argue that the errors in the derivative cancel out on average (plausible, since you're integrating) and the deviation is bounded over all of $R$.  
> **username_2**: Note that you might want to use a different loss function here than in a standard setting as error "accumulates" when you integrate. Particularly, points near the base point of integration matter more than points far from it. And you don't want to do catastrophically badly on a small set of inputs, whereas in a normal setting that's acceptable.  

---

