<p>Check out the book <a href="http://rads.stackoverflow.com/amzn/click/0932633420">The Psychology of Computer Programming by Gerald Weinberg</a>.  Originally published in 1971 it was recently produced in a Silver Anniversary version.  I own an original copy and read it again about every five years. Weinberg and Glenford Myers both wrote amazing texts which will be useful forever.</p>

<p>In Weinberg, the situation you are describing is specifically addressed, and his explanation says it is the result of knocking down preset pathways cognitively, then when you have to reestablish them you have cleared some cognitive assumptions placed there in the initial thinking of the problem. He also identifies the simple event where you begin to explain the problem to another only to suddenly discover the solution whereas it eluded you for hours before.  This, again, results from employing different neuronal circuitry when we speak out loud versus when we are doing slow thinking (a la Daniel Kahneman, Thinking Fast and Slow).  Weinberg's work is <strong>full</strong> of fun explanations and examples of how weird software engineering psychology can be.</p>

CC BY-SA 3.0
2012-01-30T21:44:21.350Z
2012-02-02T19:46:49.143Z
2012-02-02T19:46:49.143Z
C9Gm
<p>The phenomenon's called the incubation effect. Wikipedia operationally defines the incubation effect as any benefit of a break during problem solving. In Wallas’ (1926) four-stage model of innovative problem solving or creativity, the incubation stage is the stage in which one takes some time away from the problem (the stages are: preparation, incubation, illumination, verification).</p>

<p>I too oftentimes experience this phenomenon while programming.</p>

<p>To start answering your question about how the incubation effect works, I'll point to Segal's attention withdrawal and returning act hypotheses presented in the (2004) paper entitled <a href="http://www.tandfonline.com/doi/abs/10.1207/s15326934crj1601_13">Incubation in Insight Problem Solving</a>. </p>

<blockquote>
  <p>To solve an insight puzzle [or a programming problem that one's stuck on], the solver must escape
  from the mental activity governed by the false assumption
  inside the deficient problem space. That could be
  achieved by a delicate mental condition that allows, on
  one hand, a withdrawal of the attention governed by
  the false assumption, and, on the other hand, some
  level of activation of the problem’s elements in the
  solver’s mind. The presence of the problem’s elements
  in the mind—not anymore constrained by the relations
  forced by the false assumption—provides the solver an
  opportunity to apply another structure to these elements,
  this time governed by a different assumption
  that may lead to the right solution. (p. 143)</p>
</blockquote>

<p>I'll then refer you to <a href="https://docs.google.com/open?id=1xoJoE9l2SKKqF8pSEX5QKnxDZptcbuSSROZGpRm-nioetmBfn1K2PMdvmHHR">this essay of mine</a> for an exposition and critical analysis of the above-mentioned ideas, and for much much more. Feedback on the essay is welcome.</p>

<p>To address the quote from Kandell in the OP, here are some (paraphrased) point-form notes from PSY370 at University of Toronto on November 17, 2011. (Parts of the following might be difficult for you to understand; read some research, and take the class! :p). </p>

<blockquote>
  <p>•  Is creativity just insight?</p>
  
  <blockquote>
    <p>•    If yes and we can explain insight, then it would make creativity scientifically tractable.</p>
    
    <blockquote>
      <p>o   Weisberg wants to make that move.</p>
    </blockquote>
    
    <p>•    Bowden proposes that there’s nothing to creativity other than insight prob solving. Computational model of creativity.</p>
  </blockquote>
  
  <p>• Bowden: a person is creative when they solve a problem that they couldn’t solve before (because they misframed it).</p>
  
  <blockquote>
    <p>•    Creativity is whatever reconstrual allows you to do something or solve some problem you couldn’t solve before. </p>
    
    <blockquote>
      <p>•   Individually/personally creative.</p>
    </blockquote>
    
    <p>•    Historically creative if individual comes up with a way of solving the problem that humans couldn’t do before.</p>
    
    <blockquote>
      <p>o   Solution that feeds into distributed cognition.</p>
      
      <p>o   E.g. Einstein.</p>
      
      <p>o   The more an act is historically creative, the more creative we tend to find it.</p>
    </blockquote>
    
    <p>•    Both historical and personal creativity are just insight.</p>
  </blockquote>
  
  <p>• (One of) Vervaeke's insight(s) about problem solving:</p>
  
  <blockquote>
    <p>•     Maybe creativity’s more about problem finding. We value both (problem finding and problem solving) for different reasons. There's a difference in how we’re motivated and valuing. Maybe to get at what’s different about creativity, we have to get at motivation and valuation. Insight: cognitive work. Creativity: cognitive play. The difference between work and play isn’t so much what we’re doing, but the context of motivation (what goals we’re pursuing). This is a fundamental difference between insight and creativity.</p>
  </blockquote>
</blockquote>

<p>...</p>

<blockquote>
  <p>•  In one sense, insight and creativity are fundamentally the same, but in another sense they’re fundamentally different.</p>
  
  <blockquote>
    <p>•    In insight we do learning-to-learn for the sake of improving creativity. In creativity, we put ourselves in a learning situation for the sake of improving learning-to-learn.</p>
  </blockquote>
  
  <p>•  Capacity for creativity is specific improvement for our insight.</p>
  
  <blockquote>
    <p>•    Ability to think projectively.</p>
  </blockquote>
  
  <p>•  No extra machinery to creativity other than insight.</p>
  
  <blockquote>
    <p>•    But different purpose.</p>
    
    <blockquote>
      <p>o   Instead of learning-to-learn for the sake of learning, we put ourselves in a learning context for the sake of learning-to-learn.</p>
    </blockquote>
    
    <p>•    Therefore, we can explain creativity with the theoretical machinery of insight but also explain the purpose of creativity.</p>
    
    <p>•    Explain mutual reinforcement of creativity and insight, like wake-sleep [re: Hinton] reinforcement.</p>
  </blockquote>
</blockquote>

<p>As for an explanation of what insight is and how it works, I begin by endorsing Stephen and Dixon’s (2009, p. 94; italics mine) account. </p>

<blockquote>
  <p>We propose that the theory of open, nonlinear systems (see Ebeling &amp; Sokolov, 2005; Hilborn, 1994; Klimontovich, 1991 for further discussion of this theory) is of utmost relevance to problem solving. It provides a compelling account of the unfolding of cognition in action and the phenomenon of insight. The exchange between an open, nonlinear system and its environment leads to complex interactions and energy flows. These interactions and flows amount to perturbations with which the system must cope. The solution to such perturbations is the self-organization of new steady states. Action is the complex interface at which the cognitive system and environment meet. The self-organizing steady states are cognitive structures that emerge as a result of exchanges between the cognitive system and environment. <em>Insight is thus emergent structure forged amidst the nonlinear interactions of cognition, action, and the environment.</em> </p>
</blockquote>

<p>The dynamical systems phenomenon of self-organizing criticality (discovered by Bak, Tang, and Wisenfeld, 1987, 1988) can ground the theory of self-organized steady states in neural dynamics (Vervaeke, PSY370, Nov. 10, 2011; cf. Irving, Vervaeke, and Ferraro, 2010). This grounding greatly increases the plausibility of Stephen and Dixon’s (2009) account. The grounding of self-organized steady states, combined with acceptance of Stephen and Dixon’s (2009) account and acceptance of Schilling’s “small-world network model of cognitive insight” (2005, p. 1), leads to the claim that “insight is a process of self-organizing criticality that affords re-construal” (Vervaeke, PSY370, Nov. 10, 2011), since the process of self-organizing criticality helps to make small-world networks that cause re-construal by shifting back and forth between synchronous and asynchronous firing. </p>

<p>Insight is not computation. It’s a dynamical self-organizing system that makes small-world networks that cause reconstrual. We're now past the search-inference framework and the gestalt framework. Insight is developmentally running between perception and action: termed enaction. It's a dynamical system running on perception and action through recursive internal mutual modelling. The cerebellum is running on parts of brain that are acting and perceiving. (Vervaeke, PSY370, Nov. 10, '11).</p>

<p>I'm going to hand-wave and just say that good arguments for the cerebellum being integral to both insight and creativity were presented in that class.</p>

<p>See my essay for references.</p>

CC BY-SA 3.0
2012-10-31T08:10:36.690Z
Note: stackexchange prevented me from hyperlinking to more research. Vote this answer up so that I can do things like that!
Daniel
CC BY-SA 3.0
2012-10-31T08:10:55.770Z
Someone with sufficient privileges should add the tags 'insight,' 'incubation,' and 'creativity.'
Daniel
CC BY-SA 3.0
2012-10-31T12:32:17.860Z
I've given you an upvote Daniel -- thank you for the thorough answer! I don't believe that *incubation* is a large enough concept to warrant it's own tag, and we don't have an *insight* or a *creativity* tag used anywhere else, so I'm not sure we need to create those...
Josh
CC BY-SA 3.0
2012-10-31T22:54:39.857Z
Thank you Josh. Much appreciated. Since Google scholar gives 16,800 results for the search incubation insight cognit, it seems fair to say that incubation's a largish concept. Insight, creativity, and incubation are concepts that have been subject to serious (empirical) research published in respected journals, and are of great interest to many cognitive scientists, as they're central to thinking. I still think that we should start using those tags, as they might help searchers, but it's no biggie so I won't push further.
Daniel
CC BY-SA 3.0
2012-11-01T03:45:14.370Z
Typically tags on a Stack Exchange site relate to topics for which there are a lot of questions on that site, or topics which users would want to follow. That's sortof what I meant above... but tags are *not* my strongest area as a mod, and we have some users who are very adept at taxonomy. Please feel free to [ask on Meta](http://meta.cogsci.stackexchange.com/) if those tags should be added Daniel!
Josh
CC BY-SA 3.0
2014-02-28T22:05:49.147Z
Well, now we have a [tag:creatvity] tag, but that's how it's spelled, unfortunately...
Nick Stauner
CC BY-SA 3.0
2012-10-31T08:10:00.403Z
2012-10-31T08:27:32.267Z
2012-10-31T08:27:32.267Z
Daniel
<p>What you are describing is a mental bottleneck (or that is the term coined by David Rock Anyway - I actually wrote about it (in the context of Dual N Back training <a href="http://profitawareness.com/blog/2011/11/dual-n-back-training-to-break-mental-bottlenecks/">in this blog post</a> - to quote myself </p>

<blockquote>
  <p>A bottleneck happens when you can't solve a problem because you can't
  remove a bad thought from your brain.  A bad thought is something you
  know doesn't work, but you keep thinking about anyway, simple because
  your brain lacks the energy to get rid of it and replace it with
  something new.</p>
</blockquote>

<p>Essentially by taking a break from the problem you are allowing the bad thoughts to leave your brain, quite similar to rebooting a computer.  Once the bad thoughts are gone you can load everything fresh and your brain is no longer burdened by inhibiting the bad thoughts, so you can devote more brainpower to solutions.</p>

CC BY-SA 3.0
2012-01-18T22:29:36.150Z
2012-01-18T22:29:36.150Z
Steve French
<p>I believe that <a href="https://skeptics.stackexchange.com/a/4319/561">this</a> excellent answer can shed some light on why you might be unable to figure out the problem. The answerer described it as mental exhaustion, mental fatigue or mental stress. This, though does not explain why you will suddenly see the solution, but I would guess that the tired part of the brain will continue working on the problem in some way.</p>

CC BY-SA 3.0
2012-01-18T20:50:45.610Z
That's very interesting, thanks. I am still looking for something that actually explains how the answer comes to me, even when I'm not actively thinking about the problem.
Josh
CC BY-SA 3.0
2012-01-18T20:34:32.127Z
2012-01-18T20:54:02.517Z
2017-04-13T12:46:43.487Z
David
<p>It sounds like you're talking about a classic example of <a href="http://en.wikipedia.org/wiki/Incubation_%28psychology%29" rel="noreferrer"><strong>Incubation</strong></a>.</p>

<blockquote>
  <p>Incubation is defined as a process of unconscious recombination of thought elements that were stimulated through conscious work at one point in time, resulting in novel ideas at some later point in time.</p>
</blockquote>

<p>Here's a great article by John F. Kihlstrom: <a href="http://ist-socrates.berkeley.edu/~kihlstrm/Underwood96.htm" rel="noreferrer">Intuition, Incubation, and Insight: Implicit Cognition in Problem Solving</a>. Basically it is believed that Incubation or stopping conscious thought on a problem allows one to find more creative solutions to a problem:</p>

<blockquote>
  <p>In these cases, Wallas argued, thinkers enter an incubation stage in which they no longer consciously think about the problem. Wallas (1926) actually distinguished this phenomenon between two forms of incubation: "the period of abstention may be spent either in conscious mental work on other problems, or in a relaxation from all conscious mental work" (p. 86). </p>
  
  <p>Wallas believed that there might be certain economies of thought achieved by leaving certain problems unfinished while working on others, but he also believed that solutions achieved by this approach suffered in depth and richness. In many cases of difficult and complex creative thought, he believed, deeper and richer solutions could be achieved by a suspension of conscious thought altogether, permitting "the free working of the unconscious or partially conscious processes of the mind" (p. 87).1 In either case, Wallas noted that the incubation period was often followed by the illumination stage, the "flash" (p. 93) in which the answer appears in the consciousness of the thinker.</p>
</blockquote>

<p>Kihlstrom's references contain many good experiments backing up the claims made.</p>

<p>A reason incubation may work is because it releases "<strong>fixation</strong>"; that case of being "stuck" which is a sort of mental rut which prevents one from thinking of new answers or methods of solving a problem. </p>

<p>We become stuck on an idea that we believe <em>should</em> work but doesn't, which may hold us back from thinking of a different solution which actually does work, one we may have previously not considered or disregarded.</p>

<p>A great dissertation by Bo T. Christensen 
 covers both ideas of Fixation and Incubation in depth: <a href="http://www.interruptions.net/literature/Christensen-Dissertation.pdf" rel="noreferrer">Creative Cognition: Analogy and Incubation.</a> </p>

CC BY-SA 3.0
2014-07-17T13:35:19.267Z
Conscious focus blocks consciousness from receiving messages from the unconscious. Stopping the conscious focus, removes this bias and allows it to hear what the unconscious was communicating.
please delete me
CC BY-SA 3.0
2012-07-06T07:08:05.693Z
@BenBrocka, link broken
JeroenEijkhof
CC BY-SA 3.0
2012-01-18T20:52:35.697Z
Ah ha, this may be the very concept I am wondering about!
Josh
CC BY-SA 3.0
2012-01-18T20:57:03.493Z
@Josh just added some more information about Fixation as well, which is also very relevant, and another great article on the matter.
Zelda
CC BY-SA 3.0
2012-01-19T07:09:01.140Z
+1 Sounds good. Wonder why it is called *incubation* though, as it does not seem to resemble the general literary meaning of the word.
Kris
CC BY-SA 3.0
2012-01-19T14:11:53.423Z
@Kris how so? Sitting on a thing (idea) until it develops seems like an apt metaphor.
Zelda
CC BY-SA 3.0
2012-01-19T15:00:16.887Z
Which is what my objection was. Isn't the OP about NOT sitting on it but staying away from it for a while?
Kris
CC BY-SA 3.0
2012-01-19T15:14:54.147Z
Your mind sits on the egg passively. Incubation is (arguably) a passive process in this context. Tearing open the egg would be actively seeking it, and would yield less than ideal results.
Zelda
CC BY-SA 3.0
2012-01-19T15:16:09.427Z
LMAO @Ben, "less than ideal results", that's great!
Josh
CC BY-SA 3.0
2012-02-03T22:26:20.033Z
An interesting contribution from a redditor:http://www.psych.lancs.ac.uk/people/uploads/tomormerod20090227T152723.pdf
Zelda
CC BY-SA 4.0
2012-01-18T20:50:22.583Z
2018-07-16T10:33:30.110Z
2018-07-16T10:33:30.110Z
Zelda
<p>As a computer programmer, I have noticed an interesting phenomenon: If I am stuck on a particular problem in my work, often if I stop thinking about the problem and do something else, the answer will suddenly come to me.</p>

<p>Is there a name for this phenomenon? How does this work? Has any research been done on this? How is it that taking a break from a problem sometimes allows you to figure out the answer?</p>

<p><strong>Edit:</strong> I remember now where I heard about this phenomenon: on the <a href="http://www.charlierose.com/view/interview/12025">Charlie Rose Brain Series</a>, Eric Kandel of Colombia University says (at 43:20 in)</p>

<blockquote>
  <p>[The unconscious] can do many processes at the same time. You can either focus on one
  thing  or another, you can't focus on two or three things at the same time. Because
  consciousness is very limited in what it can do; unconsciousness is much broader. And
  although we know very little about the true nature of creativity, one emerging theme that
  comes out of this is that, if you're trying to solve a mathematical problem... if
  you're trying to solve any intellectual problem... you keep on focusing at it, you may
  get stuck. Taking a break, taking a shower, going for a walk, playing golf, you come
  back refreshed. And often doing the other activity, <em>boom</em>, the idea will come to you.</p>
</blockquote>

<p>But then they change the subject! What is the concept he's talking about, and where can I read more about how it works?</p>

CC BY-SA 3.0
2014-04-03T21:10:57.200Z
I know a similar phenomenon - that you remember unsolved problems or uncompleted tasks better than solved resp. completed ones - as the [Zeigarnik effect](http://en.wikipedia.org/wiki/Bluma_Zeigarnik#The_Zeigarnik_Effect). But "[there is controversy regarding the reliability of the Zeigarnik effect](http://human-factors.arc.nasa.gov/flightcognition/Publications/JEPapplied03.pdf)".
Hans-Peter Stricker
CC BY-SA 3.0
2014-11-25T16:01:44.970Z
In lucid dream induction there's a technique called "non-induction", where after a few intense days of affirmations and other training, the person completely lets go and does not even allow the topic of induction to enter one's mind. It is remarkably effective at inducing lucid dreams when properly executed. I've compared this phenomenon to a coiled spring - a spring will not expand as long as you keep compressing it. In case of induction, learning, focus and intention is the equivalent of compression
Alex Stone
CC BY-SA 3.0
2017-09-15T08:24:04.920Z
Let me share this, as you asked, At my work place when I am stuck in some issue in code or any other technical aspect, most of the solutions I end up finding in the restroom, while relaxing, its like all the probabilities and possibilities shows up and the issues are narrowed down to a single point. And in a while ,I know that where the problem is, and what will be the solution. Cheers that you asked. So me. :P
nobalG
CC BY-SA 3.0
2012-06-14T13:51:19.703Z
I quite often find myself stuck on problems when doing programming. I really try to concentrate, tries to visualize the problem, writes code, etc. to find a solution to the problem. After a while I'm so frustrated not finding a solution that I'll start working with another part of the software. When doing this context switch I very often come up with solutions to the original problem which makes me directly jump back and write code for that (which makes me forget about the part I was currently working on...) I still haven't found a good method to manage lots of these context switches though...
Erik Hellström
CC BY-SA 3.0
2012-02-03T09:02:49.550Z
I'd like to share an old take on this. The 19th century mathematician Henri Poincaré has described something very similar in his essay Mathematical Creation. So maybe that is just how people think. (I have this story and the quiotation from the book [Letters to a young mathematician](http://books.google.cz/books?id=FOkPrY-HQVQC&lpg=PP1&hl=cs&pg=PA54#v=onepage&q&f=false) (link points to the right page in a preview on Google Books) by Ian Steward)
user7610
<terminology><problem-solving><unconscious><creativity>
How is it that taking a break from a problem sometimes allows you to figure out the answer?
Josh
# How is it that taking a break from a problem sometimes allows you to figure out the answer?

(Asked by: username_0 on 2012-01-18)

As a computer programmer, I have noticed an interesting phenomenon: If I am stuck on a particular problem in my work, often if I stop thinking about the problem and do something else, the answer will suddenly come to me.

Is there a name for this phenomenon? How does this work? Has any research been done on this? How is it that taking a break from a problem sometimes allows you to figure out the answer?

**Edit:** I remember now where I heard about this phenomenon: on the [Charlie Rose Brain Series](http://www.charlierose.com/view/interview/12025), Eric Kandel of Colombia University says (at 43:20 in)

> [The unconscious] can do many processes at the same time. You can either focus on one
> thing or another, you can't focus on two or three things at the same time. Because
> consciousness is very limited in what it can do; unconsciousness is much broader. And
> although we know very little about the true nature of creativity, one emerging theme that
> comes out of this is that, if you're trying to solve a mathematical problem... if
> you're trying to solve any intellectual problem... you keep on focusing at it, you may
> get stuck. Taking a break, taking a shower, going for a walk, playing golf, you come
> back refreshed. And often doing the other activity, *boom*, the idea will come to you.

But then they change the subject! What is the concept he's talking about, and where can I read more about how it works?

**Tags:** terminology, problem-solving, unconscious, creativity

> **Comments:**
> **username_5**: I'd like to share an old take on this. The 19th century mathematician Henri Poincaré has described something very similar in his essay Mathematical Creation. So maybe that is just how people think. (I have this story and the quiotation from the book [Letters to a young mathematician](http://books.google.cz/books?id=FOkPrY-HQVQC&lpg=PP1&hl=cs&pg=PA54#v=onepage&q&f=false) (link points to the right page in a preview on Google Books) by Ian Steward)  
> **username_4**: I quite often find myself stuck on problems when doing programming. I really try to concentrate, tries to visualize the problem, writes code, etc. to find a solution to the problem. After a while I'm so frustrated not finding a solution that I'll start working with another part of the software. When doing this context switch I very often come up with solutions to the original problem which makes me directly jump back and write code for that (which makes me forget about the part I was currently working on...) I still haven't found a good method to manage lots of these context switches though...  
> **username_1**: I know a similar phenomenon - that you remember unsolved problems or uncompleted tasks better than solved resp. completed ones - as the [Zeigarnik effect](http://en.wikipedia.org/wiki/Bluma_Zeigarnik#The_Zeigarnik_Effect). But "[there is controversy regarding the reliability of the Zeigarnik effect](http://human-factors.arc.nasa.gov/flightcognition/Publications/JEPapplied03.pdf)".  
> **username_2**: In lucid dream induction there's a technique called "non-induction", where after a few intense days of affirmations and other training, the person completely lets go and does not even allow the topic of induction to enter one's mind. It is remarkably effective at inducing lucid dreams when properly executed. I've compared this phenomenon to a coiled spring - a spring will not expand as long as you keep compressing it. In case of induction, learning, focus and intention is the equivalent of compression  
> **username_3**: Let me share this, as you asked, At my work place when I am stuck in some issue in code or any other technical aspect, most of the solutions I end up finding in the restroom, while relaxing, its like all the probabilities and possibilities shows up and the issues are narrowed down to a single point. And in a while ,I know that where the problem is, and what will be the solution. Cheers that you asked. So me. :P  

---

## 5 answers

### Answer by username_11 on 2012-01-18. Score: 142 | Accepted answer

It sounds like you're talking about a classic example of [**Incubation**](http://en.wikipedia.org/wiki/Incubation_%28psychology%29).

> Incubation is defined as a process of unconscious recombination of thought elements that were stimulated through conscious work at one point in time, resulting in novel ideas at some later point in time.

Here's a great article by John F. Kihlstrom: [Intuition, Incubation, and Insight: Implicit Cognition in Problem Solving](http://ist-socrates.berkeley.edu/~kihlstrm/Underwood96.htm). Basically it is believed that Incubation or stopping conscious thought on a problem allows one to find more creative solutions to a problem:

> In these cases, Wallas argued, thinkers enter an incubation stage in which they no longer consciously think about the problem. Wallas (1926) actually distinguished this phenomenon between two forms of incubation: "the period of abstention may be spent either in conscious mental work on other problems, or in a relaxation from all conscious mental work" (p. 86).
>
> Wallas believed that there might be certain economies of thought achieved by leaving certain problems unfinished while working on others, but he also believed that solutions achieved by this approach suffered in depth and richness. In many cases of difficult and complex creative thought, he believed, deeper and richer solutions could be achieved by a suspension of conscious thought altogether, permitting "the free working of the unconscious or partially conscious processes of the mind" (p. 87).1 In either case, Wallas noted that the incubation period was often followed by the illumination stage, the "flash" (p. 93) in which the answer appears in the consciousness of the thinker.

Kihlstrom's references contain many good experiments backing up the claims made.

A reason incubation may work is because it releases "**fixation**"; that case of being "stuck" which is a sort of mental rut which prevents one from thinking of new answers or methods of solving a problem.

We become stuck on an idea that we believe *should* work but doesn't, which may hold us back from thinking of a different solution which actually does work, one we may have previously not considered or disregarded.

A great dissertation by Bo T. Christensen
covers both ideas of Fixation and Incubation in depth: [Creative Cognition: Analogy and Incubation.](http://www.interruptions.net/literature/Christensen-Dissertation.pdf)

> **Comments:**
> **username_0**: Ah ha, this may be the very concept I am wondering about!  
> **username_11**: @username_0 just added some more information about Fixation as well, which is also very relevant, and another great article on the matter.  
> **username_14**: +1 Sounds good. Wonder why it is called *incubation* though, as it does not seem to resemble the general literary meaning of the word.  
> **username_11**: @username_14 how so? Sitting on a thing (idea) until it develops seems like an apt metaphor.  
> **username_14**: Which is what my objection was. Isn't the OP about NOT sitting on it but staying away from it for a while?  
> **username_11**: Your mind sits on the egg passively. Incubation is (arguably) a passive process in this context. Tearing open the egg would be actively seeking it, and would yield less than ideal results.  
> **username_0**: LMAO @Ben, "less than ideal results", that's great!  
> **username_11**: An interesting contribution from a redditor:http://www.psych.lancs.ac.uk/people/uploads/tomormerod20090227T152723.pdf  
> **username_13**: @BenBrocka, link broken  
> **username_12**: Conscious focus blocks consciousness from receiving messages from the unconscious. Stopping the conscious focus, removes this bias and allows it to hear what the unconscious was communicating.  

---

### Answer by username_6 on 2012-01-30. Score: 29

Check out the book [The Psychology of Computer Programming by Gerald Weinberg](http://rads.stackoverflow.com/amzn/click/0932633420). Originally published in 1971 it was recently produced in a Silver Anniversary version. I own an original copy and read it again about every five years. Weinberg and Glenford Myers both wrote amazing texts which will be useful forever.

In Weinberg, the situation you are describing is specifically addressed, and his explanation says it is the result of knocking down preset pathways cognitively, then when you have to reestablish them you have cleared some cognitive assumptions placed there in the initial thinking of the problem. He also identifies the simple event where you begin to explain the problem to another only to suddenly discover the solution whereas it eluded you for hours before. This, again, results from employing different neuronal circuitry when we speak out loud versus when we are doing slow thinking (a la username_7 Kahneman, Thinking Fast and Slow). Weinberg's work is **full** of fun explanations and examples of how weird software engineering psychology can be.


---

### Answer by username_7 on 2012-10-31. Score: 18

The phenomenon's called the incubation effect. Wikipedia operationally defines the incubation effect as any benefit of a break during problem solving. In Wallas’ (1926) four-stage model of innovative problem solving or creativity, the incubation stage is the stage in which one takes some time away from the problem (the stages are: preparation, incubation, illumination, verification).

I too oftentimes experience this phenomenon while programming.

To start answering your question about how the incubation effect works, I'll point to Segal's attention withdrawal and returning act hypotheses presented in the (2004) paper entitled [Incubation in Insight Problem Solving](http://www.tandfonline.com/doi/abs/10.1207/s15326934crj1601_13).

> To solve an insight puzzle [or a programming problem that one's stuck on], the solver must escape
> from the mental activity governed by the false assumption
> inside the deficient problem space. That could be
> achieved by a delicate mental condition that allows, on
> one hand, a withdrawal of the attention governed by
> the false assumption, and, on the other hand, some
> level of activation of the problem’s elements in the
> solver’s mind. The presence of the problem’s elements
> in the mind—not anymore constrained by the relations
> forced by the false assumption—provides the solver an
> opportunity to apply another structure to these elements,
> this time governed by a different assumption
> that may lead to the right solution. (p. 143)

I'll then refer you to [this essay of mine](https://docs.google.com/open?id=1xoJoE9l2SKKqF8pSEX5QKnxDZptcbuSSROZGpRm-nioetmBfn1K2PMdvmHHR) for an exposition and critical analysis of the above-mentioned ideas, and for much much more. Feedback on the essay is welcome.

To address the quote from Kandell in the OP, here are some (paraphrased) point-form notes from PSY370 at University of Toronto on November 17, 2011. (Parts of the following might be difficult for you to understand; read some research, and take the class! :p).

> • Is creativity just insight?
>
> > • If yes and we can explain insight, then it would make creativity scientifically tractable.
> >
> > > o Weisberg wants to make that move.
> >
> > • Bowden proposes that there’s nothing to creativity other than insight prob solving. Computational model of creativity.
>
> • Bowden: a person is creative when they solve a problem that they couldn’t solve before (because they misframed it).
>
> > • Creativity is whatever reconstrual allows you to do something or solve some problem you couldn’t solve before.
> >
> > > • Individually/personally creative.
> >
> > • Historically creative if individual comes up with a way of solving the problem that humans couldn’t do before.
> >
> > > o Solution that feeds into distributed cognition.
> > >
> > > o E.g. Einstein.
> > >
> > > o The more an act is historically creative, the more creative we tend to find it.
> >
> > • Both historical and personal creativity are just insight.
>
> • (One of) Vervaeke's insight(s) about problem solving:
>
> > • Maybe creativity’s more about problem finding. We value both (problem finding and problem solving) for different reasons. There's a difference in how we’re motivated and valuing. Maybe to get at what’s different about creativity, we have to get at motivation and valuation. Insight: cognitive work. Creativity: cognitive play. The difference between work and play isn’t so much what we’re doing, but the context of motivation (what goals we’re pursuing). This is a fundamental difference between insight and creativity.

...

> • In one sense, insight and creativity are fundamentally the same, but in another sense they’re fundamentally different.
>
> > • In insight we do learning-to-learn for the sake of improving creativity. In creativity, we put ourselves in a learning situation for the sake of improving learning-to-learn.
>
> • Capacity for creativity is specific improvement for our insight.
>
> > • Ability to think projectively.
>
> • No extra machinery to creativity other than insight.
>
> > • But different purpose.
> >
> > > o Instead of learning-to-learn for the sake of learning, we put ourselves in a learning context for the sake of learning-to-learn.
> >
> > • Therefore, we can explain creativity with the theoretical machinery of insight but also explain the purpose of creativity.
> >
> > • Explain mutual reinforcement of creativity and insight, like wake-sleep [re: Hinton] reinforcement.

As for an explanation of what insight is and how it works, I begin by endorsing Stephen and Dixon’s (2009, p. 94; italics mine) account.

> We propose that the theory of open, nonlinear systems (see Ebeling & Sokolov, 2005; Hilborn, 1994; Klimontovich, 1991 for further discussion of this theory) is of utmost relevance to problem solving. It provides a compelling account of the unfolding of cognition in action and the phenomenon of insight. The exchange between an open, nonlinear system and its environment leads to complex interactions and energy flows. These interactions and flows amount to perturbations with which the system must cope. The solution to such perturbations is the self-organization of new steady states. Action is the complex interface at which the cognitive system and environment meet. The self-organizing steady states are cognitive structures that emerge as a result of exchanges between the cognitive system and environment. *Insight is thus emergent structure forged amidst the nonlinear interactions of cognition, action, and the environment.*

The dynamical systems phenomenon of self-organizing criticality (discovered by Bak, Tang, and Wisenfeld, 1987, 1988) can ground the theory of self-organized steady states in neural dynamics (Vervaeke, PSY370, Nov. 10, 2011; cf. Irving, Vervaeke, and Ferraro, 2010). This grounding greatly increases the plausibility of Stephen and Dixon’s (2009) account. The grounding of self-organized steady states, combined with acceptance of Stephen and Dixon’s (2009) account and acceptance of Schilling’s “small-world network model of cognitive insight” (2005, p. 1), leads to the claim that “insight is a process of self-organizing criticality that affords re-construal” (Vervaeke, PSY370, Nov. 10, 2011), since the process of self-organizing criticality helps to make small-world networks that cause re-construal by shifting back and forth between synchronous and asynchronous firing.

Insight is not computation. It’s a dynamical self-organizing system that makes small-world networks that cause reconstrual. We're now past the search-inference framework and the gestalt framework. Insight is developmentally running between perception and action: termed enaction. It's a dynamical system running on perception and action through recursive internal mutual modelling. The cerebellum is running on parts of brain that are acting and perceiving. (Vervaeke, PSY370, Nov. 10, '11).

I'm going to hand-wave and just say that good arguments for the cerebellum being integral to both insight and creativity were presented in that class.

See my essay for references.

> **Comments:**
> **username_7**: Note: stackexchange prevented me from hyperlinking to more research. Vote this answer up so that I can do things like that!  
> **username_7**: Someone with sufficient privileges should add the tags 'insight,' 'incubation,' and 'creativity.'  
> **username_0**: I've given you an upvote username_7 -- thank you for the thorough answer! I don't believe that *incubation* is a large enough concept to warrant it's own tag, and we don't have an *insight* or a *creativity* tag used anywhere else, so I'm not sure we need to create those...  
> **username_7**: Thank you username_0. Much appreciated. Since Google scholar gives 16,800 results for the search incubation insight cognit, it seems fair to say that incubation's a largish concept. Insight, creativity, and incubation are concepts that have been subject to serious (empirical) research published in respected journals, and are of great interest to many cognitive scientists, as they're central to thinking. I still think that we should start using those tags, as they might help searchers, but it's no biggie so I won't push further.  
> **username_0**: Typically tags on a Stack Exchange site relate to topics for which there are a lot of questions on that site, or topics which users would want to follow. That's sortof what I meant above... but tags are *not* my strongest area as a mod, and we have some users who are very adept at taxonomy. Please feel free to [ask on Meta](http://meta.cogsci.stackexchange.com/) if those tags should be added username_7!  
> **username_8**: Well, now we have a [tag:creatvity] tag, but that's how it's spelled, unfortunately...  

---

### Answer by username_9 on 2012-01-18. Score: 12

What you are describing is a mental bottleneck (or that is the term coined by username_10 Rock Anyway - I actually wrote about it (in the context of Dual N Back training [in this blog post](http://profitawareness.com/blog/2011/11/dual-n-back-training-to-break-mental-bottlenecks/) - to quote myself

> A bottleneck happens when you can't solve a problem because you can't
> remove a bad thought from your brain. A bad thought is something you
> know doesn't work, but you keep thinking about anyway, simple because
> your brain lacks the energy to get rid of it and replace it with
> something new.

Essentially by taking a break from the problem you are allowing the bad thoughts to leave your brain, quite similar to rebooting a computer. Once the bad thoughts are gone you can load everything fresh and your brain is no longer burdened by inhibiting the bad thoughts, so you can devote more brainpower to solutions.


---

### Answer by username_10 on 2012-01-18. Score: 9

I believe that [this](https://skeptics.stackexchange.com/a/4319/561) excellent answer can shed some light on why you might be unable to figure out the problem. The answerer described it as mental exhaustion, mental fatigue or mental stress. This, though does not explain why you will suddenly see the solution, but I would guess that the tired part of the brain will continue working on the problem in some way.

> **Comments:**
> **username_0**: That's very interesting, thanks. I am still looking for something that actually explains how the answer comes to me, even when I'm not actively thinking about the problem.  

---

