<h3>Exogenous Variation</h3>

<p>As Andre says, you are after <em>causality</em>, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.</p>

<p>There may be salvation, however. You are after the "<strong>treatment of the treated</strong>" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less. </p>

<p>In order to remove this self selection, you could instead look at something like the "<strong>average treatment effect</strong>". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?</p>

<h3>IV Estimation</h3>

<p>Now, you will either randomly inform individuals about SO (something that StackExchange as a company might actually do), or you/they might already have data on "being informed about SO", lets call that <em>inform</em>. As long as <em>inform</em> does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become <em>inform</em>. </p>

<p>Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.</p>

<p>Then, all you have to do is regress some measure of productivity/success/career advancement onto <em>inform</em>, and perhaps <em>ability</em>.</p>

CC BY-SA 3.0
2015-05-12T13:23:30.203Z
I thought my answer was clear on that: the control group would be those with *inform* = 0.
FooBar
CC BY-SA 3.0
2015-05-12T13:31:40.800Z
Is your claim that the only people not aware of StackOverflow are living in North Korea? Because then half of my department (who do a lot of programming) should be deported back. Anyhow, given your comments here and in the remainder of this question, I don't think we will reach a conclusion. Perhaps we can agree to disagree and move on with our lives.
FooBar
CC BY-SA 3.0
2015-05-12T13:40:04.620Z
Knowledge of Stack Overflow is a red herring. We can simply block access to it (and indeed, unfortunately many programmers have this problem).
Sklivvz
CC BY-SA 3.0
2015-05-12T12:29:09.843Z
2015-05-12T12:38:41.897Z
2015-05-12T12:38:41.897Z
FooBar
<p>As many have mentioned it can be quantified in monetary terms. As Foobar mentions what we need is exogenous variation* in access to Stack Exchange. Here is one study design: </p>

<p>Inform and give a short introduction to random cohorts in their second year at university about StackOverflow, then come back in few years and compare grades and maybe later work outcomes (such as salary). If it is done in second year, grades from first year can be introduced as controls and to see if randomization was truly successful.</p>

<p>*Variation that is not (cor)related to other characteristics related to programming productivity.</p>

CC BY-SA 3.0
2015-05-12T19:20:49.233Z
2015-05-12T19:20:49.233Z
snoram
<p>Has Stack Overflow an overall positive impact on programmers productivity? </p>

<p>On one side, it takes time and effort to ask and answer questions -- thus decreasing productivity. </p>

<p>On the other side, more and more programmer questions are answered and easily found through Google. This allows many programmers to find solutions to their problems quicker than ever before, thus increasing productivity.</p>

<p>Has Stack Overflow been economically beneficial to programmer productivity? If so, can this be quantified in terms of money? </p>

<p>Note: I've asked this <a href="https://skeptics.stackexchange.com/questions/18539/has-stack-overflow-saved-billions-of-dollars-in-programmer-productivity">related question</a> on Skeptics.SE, but it turns out there is no peer-reviewed research on the matter.</p>

CC BY-SA 3.0
2015-05-12T11:00:48.510Z
Why do you think answering questions decreases productivity? Apart from outright obsessive checking of SO every five minutes, of course :)
Luaan
CC BY-SA 3.0
2015-05-12T12:35:27.443Z
From the linked question: *It isn't obvious how economists might measure value to an economy. It isn't obvious how computer scientists might measure productivity.* Both value to an economy and productivities are variables that we have a pretty good grip on, I would disagree on both.
FooBar
CC BY-SA 3.0
2015-05-12T13:21:16.563Z
One straightforward definition for the productivity of any worker is the value of his output (his value added), as appreciated/evaluated by the firm. This includes all within-firm externalities such as peer effects. In proper notation, with $\mathcal L$ being the set of workers at that firm, the productivity of worker $i$ would be $F(\mathcal L) - F(\mathcal L_{-i})$
FooBar
<labor-economics><productivity><software>
Has Stack Overflow been economically beneficial to programmer productivity?
Sklivvz
# Has Stack Overflow been economically beneficial to programmer productivity?

(Asked by: username_0 on 2015-05-12)

Has Stack Overflow an overall positive impact on programmers productivity?

On one side, it takes time and effort to ask and answer questions -- thus decreasing productivity.

On the other side, more and more programmer questions are answered and easily found through Google. This allows many programmers to find solutions to their problems quicker than ever before, thus increasing productivity.

Has Stack Overflow been economically beneficial to programmer productivity? If so, can this be quantified in terms of money?

Note: I've asked this [related question](https://skeptics.stackexchange.com/questions/18539/has-stack-overflow-saved-billions-of-dollars-in-programmer-productivity) on Skeptics.SE, but it turns out there is no peer-reviewed research on the matter.

**Tags:** labor-economics, productivity, software

> **Comments:**
> **username_1**: Why do you think answering questions decreases productivity? Apart from outright obsessive checking of SO every five minutes, of course :)  
> **username_2**: From the linked question: *It isn't obvious how economists might measure value to an economy. It isn't obvious how computer scientists might measure productivity.* Both value to an economy and productivities are variables that we have a pretty good grip on, I would disagree on both.  
> **username_2**: One straightforward definition for the productivity of any worker is the value of his output (his value added), as appreciated/evaluated by the firm. This includes all within-firm externalities such as peer effects. In proper notation, with $\mathcal L$ being the set of workers at that firm, the productivity of worker $i$ would be $F(\mathcal L) - F(\mathcal L_{-i})$  

---

## 2 answers

### Answer by username_2 on 2015-05-12. Score: 2

### Exogenous Variation

As Andre says, you are after *causality*, not just correlation. And the best way to find that is exogenous variation - he mentions experiments. Unless you can randomly force some people to use SO - and forbid it to a control group, there is little you can do with true experiments.

There may be salvation, however. You are after the "**treatment of the treated**" effect - those who are informed about SO, and decide to use it. That is hard to get, because the latter step will correlate with ability. Some people are just better/more efficient/smarter and hence can benefit from SO more/less.

In order to remove this self selection, you could instead look at something like the "**average treatment effect**". We inform a subsample about SO. Some will use it, some will not. What is the average increase of productivity in this sample?

### IV Estimation

Now, you will either randomly inform individuals about SO (something that StackExchange as a company might actually do), or you/they might already have data on "being informed about SO", lets call that *inform*. As long as *inform* does not correlate with relevant characteristics (such as skill), you can use that as a proxy. Unfortunately, one might argue that it actually does: Perhaps smarter/better programmers are also better at creating networks / using google, and hence are more likely to find SO and become *inform*.

Of course, if you also have data for ability, you can control for that, so this particular bias would be removed.

Then, all you have to do is regress some measure of productivity/success/career advancement onto *inform*, and perhaps *ability*.

> **Comments:**
> **username_2**: I thought my answer was clear on that: the control group would be those with *inform* = 0.  
> **username_2**: Is your claim that the only people not aware of StackOverflow are living in North Korea? Because then half of my department (who do a lot of programming) should be deported back. Anyhow, given your comments here and in the remainder of this question, I don't think we will reach a conclusion. Perhaps we can agree to disagree and move on with our lives.  
> **username_0**: Knowledge of Stack Overflow is a red herring. We can simply block access to it (and indeed, unfortunately many programmers have this problem).  

---

### Answer by username_3 on 2015-05-12. Score: 0

As many have mentioned it can be quantified in monetary terms. As Foobar mentions what we need is exogenous variation\* in access to Stack Exchange. Here is one study design:

Inform and give a short introduction to random cohorts in their second year at university about StackOverflow, then come back in few years and compare grades and maybe later work outcomes (such as salary). If it is done in second year, grades from first year can be introduced as controls and to see if randomization was truly successful.

\*Variation that is not (cor)related to other characteristics related to programming productivity.


---

