<p>To get you answer,you may wanna look at this paper  <a href="http://dept.stat.lsa.umich.edu/~gmichail/ada_final.pdf" rel="nofollow">http://dept.stat.lsa.umich.edu/~gmichail/ada_final.pdf</a>.
Algorithm 2 summarizes the step you have to take to derive a boosting algorithm from any given convex loss including logitboost. </p>

CC BY-SA 3.0
2015-01-30T10:12:24.540Z
2015-01-30T10:12:24.540Z
TNM
<p>Originally, Logitboost was derived by Friedman, Hastie, and Tibshirani (<a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.9525" rel="nofollow">paper</a>) -- their algorithm internally used a numerical procedure, via Newtons method to solve a regression problem.  </p>

<p>Later, Collins, Schapire, and Singer (<a href="http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.329" rel="nofollow">paper</a>) found an equivalent formulation, with a single-line modification from AdaBoost, setting 
$$D(i) \propto \frac{1}{1+e^{y_i f_{t-1}(x_i)}}.$$</p>

<p>Schapire has a nice summary of boosting <a href="http://www.cis.upenn.edu/~mkearns/teaching/COLT/schapire.pdf" rel="nofollow">here</a>, which also discusses Logitboost.</p>

CC BY-SA 3.0
2012-11-02T00:57:29.613Z
looks like I didn't ask the question clearly, I have cleaned up. I understand the adaboost.
add-semi-colons
CC BY-SA 3.0
2012-11-01T22:21:38.590Z
2012-11-01T22:21:38.590Z
Lev Reyzin
<p>An additive model constructed using the exponential loss function </p>

<p><span class="math-container">$$L(y, f (x)) = \exp(−yf (x))$$</span></p>

<p>gives   Adaboost. How can we derive the corresponding additive model (known as logitboost) using the logistic loss function </p>

<p><span class="math-container">$$L(y, f (x)) = \log(1 + \exp(−yf (x)))$$</span></p>

<p>What steps I should take to do the above proof?</p>

CC BY-SA 3.0
2012-11-01T22:22:51.350Z
I suggest modifying this to ask a concrete theoretical question -- e.g. "How does one minimize logistic loss...".
Lev Reyzin
<machine-learning><convex-optimization>
Derive logitboost using the logistic loss function
add-semi-colons
# Derive logitboost using the logistic loss function

(Asked by: username_0 on 2012-11-01)

An additive model constructed using the exponential loss function

$$L(y, f (x)) = \exp(−yf (x))$$

gives Adaboost. How can we derive the corresponding additive model (known as logitboost) using the logistic loss function

$$L(y, f (x)) = \log(1 + \exp(−yf (x)))$$

What steps I should take to do the above proof?

**Tags:** machine-learning, convex-optimization
> **Comments:**
> **username_1**: I suggest modifying this to ask a concrete theoretical question -- e.g. "How does one minimize logistic loss...".  

---

## 2 answers

### Answer by username_2 on 2015-01-30. Score: 2

To get you answer,you may wanna look at this paper <http://dept.stat.lsa.umich.edu/~gmichail/ada_final.pdf>.
Algorithm 2 summarizes the step you have to take to derive a boosting algorithm from any given convex loss including logitboost.


---

### Answer by username_1 on 2012-11-01. Score: 2

Originally, Logitboost was derived by Friedman, Hastie, and Tibshirani ([paper](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.9525)) -- their algorithm internally used a numerical procedure, via Newtons method to solve a regression problem.

Later, Collins, Schapire, and Singer ([paper](http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.9.329)) found an equivalent formulation, with a single-line modification from AdaBoost, setting
$$D(i) \propto \frac{1}{1+e^{y\_i f\_{t-1}(x\_i)}}.$$

Schapire has a nice summary of boosting [here](http://www.cis.upenn.edu/~mkearns/teaching/COLT/schapire.pdf), which also discusses Logitboost.

> **Comments:**
> **username_0**: looks like I didn't ask the question clearly, I have cleaned up. I understand the adaboost.  

---

