<p>I don't know if this is helpful, but I would chime in as a programmer. I would argue that few people truly know all the parts of a large system by heart, instead they know how to find out/debug what's happening within the project quickly. </p>

<p><strong>A lot of large, complex programming projects are a mess.</strong> They are difficult to understand and maintain/navigate/debug. An Apple engineer once joked: </p>

<blockquote>
  <p>"If you were as smart as you could be when you wrote the program, and it's
  twice as hard to debug something than to write it, how are
  you ever going to debug it?"</p>
</blockquote>

<p>Large projects did not originate over a single day, but may be a result of an "evolution" over a few months or years. So the basic components which are simple get burdened by exceptions, special use cases, subclasses, multiple modes of operation and precedents as you have said. </p>

<p>Additionally, programmers in general rarely write useful, clear, up to date documentation of large complex systems (there's no energy left to do this at the end).</p>

<p>It is easier for the original developer to navigate such project without having to keep all details in memory, because the programmer knows what to search for. The name of the class, instance variable or a method would bring the original creator to the proper point within the project. But as a "new guy", you do not have the luxury of remembering many "landmarks" within the project. So it takes longer for you to find the solution or understand what's happening.  </p>

CC BY-SA 3.0
2013-04-18T03:03:50.953Z
2013-04-18T03:03:50.953Z
Alex Stone
<p>The problem that you are describing appears to be a problem of <a href="http://en.wikipedia.org/wiki/Short-term_memory" rel="nofollow">short-term memory</a>. It is well known among psychologists and neuroscientists that most people can retain 7 +/- 2 numbers or words in short-term memory. For a programmer, I would describe short-term memory and the process of "recalling", as a process similar to searching in a tree structure. </p>

<p>I am a software developer too. I understand your frustration, and I assure you that most programmers have similar problems when dealing with complex projects. What I recommend you to do, is to find a good systems engineering book, and learn how to work with complex software projects. If you are working with a team, I recommend you to break apart your project into modules with high cohesion and low coupling. Assign a module to each member of the team, and workout the interfaces between the sub-systems. I also recommend you to set some kind of version control. </p>

CC BY-SA 3.0
2013-04-23T15:12:34.150Z
I believe you mean 5-9 discreet "chunks" - they don't have to be numbers or words. This is why experts learn more effectively than novices (see my answer below).
theMayer
CC BY-SA 3.0
2013-04-23T16:20:57.507Z
You are right, something like "Discrete Elements/Chunks" is probably the word that I was looking for.
Jaime Ivan Cervantes
CC BY-SA 3.0
2013-04-18T07:01:02.570Z
2013-04-18T07:01:02.570Z
Jaime Ivan Cervantes
<p>I just completed an extensive study (and corresponding lit review) of how people learn.  I think you are running into the classic difference between expert and novice users.</p>

<p><strong>Experts vs Novices</strong></p>

<p>It has been demonstrated that the knowledge structures in experts are different than those in novices.  In particular, experts' knowledge is far more structured, and they are capable of immediately connecting concepts that novices are unable to see.  (I have articles I can cite if you wish, but I'm just trying to get you on the right track here).</p>

<p>Bascially, people learn by slowly (or quickly) becoming "experts," that is, by organizing their conceptual models better around various concepts, such that the distinctions between them disappear into this web of schematic knowledge.</p>

<p>It has also been observed that novices remember learning "episodes" - e.g. hands-on exploration activities or (sometimes) lectures.  As the novice becomes an expert, the details of these individual episodes tend to fuzz together and a unified cognitive schema emerges.</p>

<p><strong>Short-term Working Memory</strong></p>

<p>As was mentioned in a previous answer, short-term memory can hold 5-9 discreet "chunks" of information.  To use programming terms, a <code>chunk</code> can either be a struct, where it represents actual data stored in working memory, or it can be a pointer to the location in long-term memory where a concept is stored.  This helps explain why experts can perform so much more quickly than novices, because their working memory can be much more efficiently utilized.</p>

<p>People rely on working memory to learn. As you learn, new information is incorporated into your existing mental model (this is known as constructivism).  This is an active process requiring the information to be reprocessed in working memory before it is stored into LTM.</p>

<p><strong>Implications</strong></p>

<p>The research above suggests that you need to take the system in pieces.  Learn what each piece does via a series of hands-on <code>episodes</code>, then move on to the next piece.  Then, come back and re-start this process.  As you do so, you will find yourself becoming more and more familiar with this complex system.</p>

CC BY-SA 3.0
2013-04-23T16:27:15.993Z
By far better description than mine on short term memory. Thanks for your answer.
Jaime Ivan Cervantes
CC BY-SA 3.0
2013-04-23T17:51:18.173Z
Thanks! Learning is a complex process, not well-understood, so any little tidbit, if used properly, should help :)
theMayer
CC BY-SA 3.0
2013-04-26T09:51:51.020Z
rmayer06, thx for great answer. Some questions. 1. Can not find decent translation to "hands-on exploration activities", would you please explain what do you mean by this? 2. >their [experts'] working memory can be much more efficiently utilized. More efficiently utilized -- because of their "knowledge is far more structured"? I.e., because of they have more experience?
Vadim Samokhin
CC BY-SA 3.0
2013-04-26T18:47:56.217Z
1. Hands-on in terms of giving students a very structured problem/scenario and asking them to answer specific questions (e.g. a laboratory), some call it problem-based learning (see papers by Hmelo-Silver).  2. The knowledge structures are more efficiently organized, and more connections are made to relevant concepts, thus making recall easier.
theMayer
CC BY-SA 3.0
2013-04-23T15:10:33.643Z
2013-04-23T15:15:42.553Z
2013-04-23T15:15:42.553Z
theMayer
<p><strong>Background:</strong> First of all, I'm not in any kind related to cognitive science, I'm a programmer. I'm not even sure that it's the right place for asking my question, but I'll try anyway. So, first of all, I'll describe the initial problem that motivated me to ask this question. There are some concrete examples.</p>

<ul>
<li><p>I join the programmers team that have rather big (but not huge) project. I'm trying to figure out how it works, how each part of the system cooperates with each other. I want to see how it works as a whole, so that I could say that I understand what's going on: what goes first, what goes next, without going into pretty much detail. I want to have this image in mind and to discuss the project with no need to diving into code, or to schemes describing the project, or any additional info. The thing that does not let me do this -- I fail to remember the system components, their execution precedence that may vary, and their interactions between each other.</p></li>
<li><p>I got explanations of some project use case. Everything seems to be clear, but there comes a time when I realize that I can not hold all this in my head, can not keep all precedence of actions, the interactions between them. The whole thing becomes blurry.</p></li>
</ul>

<p>I tend to see these cases having the same reason. As this reason results in my wish for better results achievement, I call it a problem. It may or may not be a "problem" from medical or any other point of view, but this is a problem for me anyway. So, when I talk about solving this problem, I talk about improving my results in area described above.</p>

<h2>Questions</h2>

<ul>
<li>What is this problem called where a person can not hold all the details of a large project in memory?</li>
<li>Is this a failure of memory?</li>
<li>Is the problem due to lack of intelligence? </li>
<li>Would any of the following reduce the problem: chess, poker, solving math problems, nootropics?</li>
</ul>

CC BY-SA 3.0
2013-04-14T11:14:43.263Z
sounds like "seeing the forest for the trees" or the "Gestalt" of it all.  The "big  picture".  These are all terms we use in society, anyway.  Not sure of the formal psychology term.  I think your question could be more concise and include initial research once we  find it, though.
Keegan Keplinger
CC BY-SA 3.0
2013-04-14T13:55:35.023Z
@what Thx, I do the schemes and draw the structures, but I want to operate with all this stuff in memory and do it quickly. Of course I understand that it can not be achieved in a few days, but anyway I want to shorten this time frame, to improve the ability of the details remembering, of predicting what effect one component changing would have, etc.
Vadim Samokhin
<memory><intelligence>
What causes an individual to not be able to hold all the details of a large project in memory?
Vadim Samokhin
# What causes an individual to not be able to hold all the details of a large project in memory?

(Asked by: username_0 on 2013-04-14)

**Background:** First of all, I'm not in any kind related to cognitive science, I'm a programmer. I'm not even sure that it's the right place for asking my question, but I'll try anyway. So, first of all, I'll describe the initial problem that motivated me to ask this question. There are some concrete examples.

* I join the programmers team that have rather big (but not huge) project. I'm trying to figure out how it works, how each part of the system cooperates with each other. I want to see how it works as a whole, so that I could say that I understand what's going on: what goes first, what goes next, without going into pretty much detail. I want to have this image in mind and to discuss the project with no need to diving into code, or to schemes describing the project, or any additional info. The thing that does not let me do this -- I fail to remember the system components, their execution precedence that may vary, and their interactions between each other.
* I got explanations of some project use case. Everything seems to be clear, but there comes a time when I realize that I can not hold all this in my head, can not keep all precedence of actions, the interactions between them. The whole thing becomes blurry.

I tend to see these cases having the same reason. As this reason results in my wish for better results achievement, I call it a problem. It may or may not be a "problem" from medical or any other point of view, but this is a problem for me anyway. So, when I talk about solving this problem, I talk about improving my results in area described above.

## Questions

* What is this problem called where a person can not hold all the details of a large project in memory?
* Is this a failure of memory?
* Is the problem due to lack of intelligence?
* Would any of the following reduce the problem: chess, poker, solving math problems, nootropics?

**Tags:** memory, intelligence

> **Comments:**
> **username_1**: sounds like "seeing the forest for the trees" or the "Gestalt" of it all.  The "big  picture".  These are all terms we use in society, anyway.  Not sure of the formal psychology term.  I think your question could be more concise and include initial research once we  find it, though.  
> **username_0**: @what Thx, I do the schemes and draw the structures, but I want to operate with all this stuff in memory and do it quickly. Of course I understand that it can not be achieved in a few days, but anyway I want to shorten this time frame, to improve the ability of the details remembering, of predicting what effect one component changing would have, etc.  

---

## 3 answers

### Answer by username_4 on 2013-04-23. Score: 8 | Accepted answer

I just completed an extensive study (and corresponding lit review) of how people learn. I think you are running into the classic difference between expert and novice users.

**Experts vs Novices**

It has been demonstrated that the knowledge structures in experts are different than those in novices. In particular, experts' knowledge is far more structured, and they are capable of immediately connecting concepts that novices are unable to see. (I have articles I can cite if you wish, but I'm just trying to get you on the right track here).

Bascially, people learn by slowly (or quickly) becoming "experts," that is, by organizing their conceptual models better around various concepts, such that the distinctions between them disappear into this web of schematic knowledge.

It has also been observed that novices remember learning "episodes" - e.g. hands-on exploration activities or (sometimes) lectures. As the novice becomes an expert, the details of these individual episodes tend to fuzz together and a unified cognitive schema emerges.

**Short-term Working Memory**

As was mentioned in a previous answer, short-term memory can hold 5-9 discreet "chunks" of information. To use programming terms, a `chunk` can either be a struct, where it represents actual data stored in working memory, or it can be a pointer to the location in long-term memory where a concept is stored. This helps explain why experts can perform so much more quickly than novices, because their working memory can be much more efficiently utilized.

People rely on working memory to learn. As you learn, new information is incorporated into your existing mental model (this is known as constructivism). This is an active process requiring the information to be reprocessed in working memory before it is stored into LTM.

**Implications**

The research above suggests that you need to take the system in pieces. Learn what each piece does via a series of hands-on `episodes`, then move on to the next piece. Then, come back and re-start this process. As you do so, you will find yourself becoming more and more familiar with this complex system.

> **Comments:**
> **username_3**: By far better description than mine on short term memory. Thanks for your answer.  
> **username_4**: Thanks! Learning is a complex process, not well-understood, so any little tidbit, if used properly, should help :)  
> **username_0**: rmayer06, thx for great answer. Some questions. 1. Can not find decent translation to "hands-on exploration activities", would you please explain what do you mean by this? 2. >their [experts'] working memory can be much more efficiently utilized. More efficiently utilized -- because of their "knowledge is far more structured"? I.e., because of they have more experience?  
> **username_4**: 1. Hands-on in terms of giving students a very structured problem/scenario and asking them to answer specific questions (e.g. a laboratory), some call it problem-based learning (see papers by Hmelo-Silver).  2. The knowledge structures are more efficiently organized, and more connections are made to relevant concepts, thus making recall easier.  

---

### Answer by username_2 on 2013-04-18. Score: 4

I don't know if this is helpful, but I would chime in as a programmer. I would argue that few people truly know all the parts of a large system by heart, instead they know how to find out/debug what's happening within the project quickly.

**A lot of large, complex programming projects are a mess.** They are difficult to understand and maintain/navigate/debug. An Apple engineer once joked:

> "If you were as smart as you could be when you wrote the program, and it's
> twice as hard to debug something than to write it, how are
> you ever going to debug it?"

Large projects did not originate over a single day, but may be a result of an "evolution" over a few months or years. So the basic components which are simple get burdened by exceptions, special use cases, subclasses, multiple modes of operation and precedents as you have said.

Additionally, programmers in general rarely write useful, clear, up to date documentation of large complex systems (there's no energy left to do this at the end).

It is easier for the original developer to navigate such project without having to keep all details in memory, because the programmer knows what to search for. The name of the class, instance variable or a method would bring the original creator to the proper point within the project. But as a "new guy", you do not have the luxury of remembering many "landmarks" within the project. So it takes longer for you to find the solution or understand what's happening.


---

### Answer by username_3 on 2013-04-18. Score: 3

The problem that you are describing appears to be a problem of [short-term memory](http://en.wikipedia.org/wiki/Short-term_memory). It is well known among psychologists and neuroscientists that most people can retain 7 +/- 2 numbers or words in short-term memory. For a programmer, I would describe short-term memory and the process of "recalling", as a process similar to searching in a tree structure.

I am a software developer too. I understand your frustration, and I assure you that most programmers have similar problems when dealing with complex projects. What I recommend you to do, is to find a good systems engineering book, and learn how to work with complex software projects. If you are working with a team, I recommend you to break apart your project into modules with high cohesion and low coupling. Assign a module to each member of the team, and workout the interfaces between the sub-systems. I also recommend you to set some kind of version control.

> **Comments:**
> **username_4**: I believe you mean 5-9 discreet "chunks" - they don't have to be numbers or words. This is why experts learn more effectively than novices (see my answer below).  
> **username_3**: You are right, something like "Discrete Elements/Chunks" is probably the word that I was looking for.  

---

