<p>Actually I have realized where the misunderstanding came from. In fact the implementation aforementioned specifies a cost for the general case, namely the multinational logistic regression. Therefore the implementation is deliberate and correct.</p>

<p>See the screenshot below.</p>

<p><a href="https://i.sstatic.net/WJBqB.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WJBqB.png" alt="enter image description here"></a></p>

CC BY-SA 3.0
2016-04-05T14:26:26.730Z
2016-04-05T14:26:26.730Z
valentin
<p>I have recently come across a <a href="https://github.com/hushell/drnn/tree/master/PGM/model/lr" rel="nofollow noreferrer">Matlab implementation</a> that appears to be using only the first term (i.e. in itself a product) of the typical logistic regression cost function. 
<a href="https://i.sstatic.net/cFdQ8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/cFdQ8.png" alt="enter image description here"></a></p>

<pre><code>f = - full(mean(sum(Y.*log(Yr))));
f = f + 0.5*lambda*sum(W(:).^2);
</code></pre>

<p>When I create a two class simple dataset and adjust the cost function to its traditional form I still get separation between classes albeit the vector of parameters are slightly different (for a fixed lambda regularizer in both cases).</p>

<p><a href="https://i.sstatic.net/8Oy2r.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8Oy2r.png" alt="enter image description here"></a></p>

<pre><code>f = - full(mean(sum(Y.*log(Yr) + (1-Y).*log(1-Yr))));
f = f + 0.5*lambda*sum(W(:).^2);
</code></pre>

<p>For better visualization see the following plot
<a href="https://i.sstatic.net/vRw9P.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vRw9P.png" alt="Orange due to simpler, blue due to full cost"></a> </p>

<p>Once the cost function is set, the parameters are learned using limited-memory BFGS. Gradient functions remain the same (i.e. derived from the correct classical cost function).
<a href="https://i.sstatic.net/tigsO.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/tigsO.png" alt="enter image description here"></a></p>

<p>Naturally, my question is: Has this choice been deliberate ? Is the first objective function sufficient ? Or is it just an obvious inconsistency.</p>

<machine-learning>
Can you use only the first summation term of cost function for typical logistic regression?
valentin
# Can you use only the first summation term of cost function for typical logistic regression?

(Asked by: username_0 on 2016-04-05)

I have recently come across a Matlab implementation that appears to be using only the first term (i.e. in itself a product) of the typical logistic regression cost function. 

f = - full(mean(sum(Y.*log(Yr))));
f = f + 0.5*lambda*sum(W(:).^2);

When I create a two class simple dataset and adjust the cost function to its traditional form I still get separation between classes albeit the vector of parameters are slightly different (for a fixed lambda regularizer in both cases).

f = - full(mean(sum(Y.*log(Yr) + (1-Y).*log(1-Yr))));
f = f + 0.5*lambda*sum(W(:).^2);

For better visualization see the following plot
 
Once the cost function is set, the parameters are learned using limited-memory BFGS. Gradient functions remain the same (i.e. derived from the correct classical cost function).

Naturally, my question is: Has this choice been deliberate ? Is the first objective function sufficient ? Or is it just an obvious inconsistency.

**Tags:** machine-learning

---

## 1 answers

### Answer by username_0 on 2016-04-05. Score: -1 | Accepted answer

Actually I have realized where the misunderstanding came from. In fact the implementation aforementioned specifies a cost for the general case, namely the multinational logistic regression. Therefore the implementation is deliberate and correct.

See the screenshot below.

[![enter image description here](https://i.sstatic.net/WJBqB.png)](https://i.sstatic.net/WJBqB.png)


---

