<p>I will answer this from the perspective of classical graph problems (or input/output problems): we have a network, each node gets something as input and each node must produce something as output. I guess this is closest to the world of traditional computational complexity.</p>

<p>I am certainly biased, but I think that in this setting, there <em>is</em> a simple and fairly commonly used model of distributed computing: <em>synchronous distributed algorithms</em>, with the definition that <em>running time = number of synchronous rounds</em>. In Peleg's terminology, this is the <em>LOCAL</em> model.</p>

<p>This model is nice as it has very few "moving parts", no parameters, etc. Nevertheless, it is very concrete: it makes sense to say that the running time of an algorithm is exactly 15 in this model. And you can prove unconditional, information-theoretic lower bounds: from this perspective, the distributed complexity of many graph problems (e.g., graph colouring) is fairly well-understood.</p>

<p>This model also provides a unified approach to many aspects of distributed computing:</p>

<ul>
<li>Message-passing vs. shared memory, broadcast vs. unicast: Irrelevant in this model.</li>
<li>Your real-world system is asynchronous? No problem, just plug in the $\alpha$-synchroniser. The time complexity (with suitable definitions) is essentially unaffected.</li>
<li>You'd like to have an algorithm for dynamic networks, or you'd like to recover from failures? Well, if your synchronous algorithm is deterministic, then you can use it to construct a <em>self-stabilising</em> algorithm. Again, the time complexity is essentially unaffected.</li>
</ul>

<hr>

<p>Now all this is fine as long as you study problems that are "truly distributed" in the sense that the running time of your algorithm is <em>smaller than the diameter of the graph</em>, i.e., no node needs to have full information on the structure of the graph. However, there are also many problems that are inherently global: the fastest algorithm in this model has running time that is linear in the diameter of the graph. In the study of those problems, the above model no longer makes any sense, and then we need to resort to something else. Typically, one starts to pay attention to the total number of messages or bits communicated in the network. That's one reason why we get several different models.</p>

<hr>

<p>Then of course we have the issue that the distributed computing community is actually <a href="http://dx.doi.org/10.1145/1835698.1835700">two different communities, with surprisingly few things in common</a>. If you lump together all models from two communities, it <em>will</em> certainly look a bit confusing... My answer above is related to only one half of the community; I trust others will fill in regarding the other half.</p>

CC BY-SA 2.5
2011-01-28T13:54:19.060Z
Thanks for the very informative answer. I would accept this as THE answer.
Mohammad Al-Turkistany
CC BY-SA 2.5
2010-08-25T00:23:38.533Z
If I understand this correctly, the point is that there is an elegant theory only for synchronous systems and not much else. With respect to systems other than synchronous ones, we are conflating problems/foci from two otherwise different communities, and this presents methodological issues with developing an single theory. Have I understood your arguments correctly?
Srikanth Sastry
CC BY-SA 2.5
2010-08-23T21:14:45.580Z
2010-08-23T21:24:38.973Z
2010-08-23T21:24:38.973Z
Jukka Suomela
<p>One romantic idea to capture various models of distributed computing has been through algebraic topology. The core idea is to construct simplicial complexes by letting points be process states, each labeled with a process id.  This is a <a href="http://www.springerlink.com/content/56712l0152526304/">primer</a> on the topic. The closest answer to your question has probably been touched upon by Eli gafni in his paper- Distributed computing- A glimmer of a theory. In his paper, he shows simulations how beginning with async shared memory for two-three processors(for fail stop and Byzantine)-he shows how can apply this to the message passing model. Crucial to understanding his simulations is the notion of viewing a distributed computing topologically </p>

CC BY-SA 2.5
2010-11-11T14:45:55.310Z
2010-11-11T14:45:55.310Z
kryptos
<p>I think the situation looks quite different if viewed in context: starting from the early works and impossibility results on Byzantine agreement (<a href="http://portal.acm.org/citation.cfm?doid=322186.322188" rel="nofollow">PSL80</a> <a href="http://portal.acm.org/citation.cfm?id=357172.357176" rel="nofollow">LSP82</a> <a href="http://portal.acm.org/citation.cfm?id=214121" rel="nofollow">FLP85</a>), it was clear soon that fundamental problems in distributed computing can only be solved at all with strict synchrony assumptions and a high degree of redundancy. As these unconditional theoretical resource lower bounds were considered infeasible for any practical purposes, research focused on developing more refined models that allowed evermore fine-grained trade-offs of assumptions (on timing guarantees or failure modes for example) vs. guarantees (i.e. number of simultaneous faults of what kinds on what type of components tolerated, e.g. processors, links) in order to give the system designers the tools to find the right trade-off for the system at hand. </p>

CC BY-SA 2.5
2010-08-23T22:24:24.587Z
I understand that the refined models were introduced to understand 'practical' solvability of problems in the distributed space. One would expect these fine-grained models to arrange themselves neatly into a hierarchy with respect to solvability, time complexity, and message complexity. Unfortunately, this is not the case. My question here, is what is the reason for this balkanization? If it is some attribute(s) inherent to distributed computing, then what are they?
Srikanth Sastry
CC BY-SA 2.5
2010-08-23T20:55:00.730Z
2010-08-24T07:29:15.997Z
2010-08-24T07:29:15.997Z
Martin Schwarz
<p>My take is that the abstractly-motivated Turing machine model of computation was a good approximation of technology until very recently, whereas models of distributed computing, from the get-go, have been motivated by the real world, which is always messier than abstractions.  </p>

<p>From, say, 1940-1995, the size of problem instances, the relative "unimportance" of parallelism and concurrency, and the macro-scale of computing devices, all "conspired" to keep Turing machines an excellent approximation of real-world computers.  However, once you start dealing with massive datasets, ubiquitous need for concurrency, biology through the algorithmic lens, etc., it is much less clear if there is an "intuitive" model of computation.  Perhaps problems hard in one model are not hard -- strictly less computationally complex -- in another.  So I believe that mainstream computational complexity is finally catching up (!) with distributed computing, by starting to consider multiple models of computation and data structures, motivated by real-world considerations.</p>

CC BY-SA 2.5
2010-08-25T02:53:51.983Z
Also consider the defining questions of the respective fields.  "Assume you can compute perfectly.  What are the limits of what you can and can't do?" vs. "Assume you have a faulty channel, processor, or assume you have an adversary.  How can you compute successfully when faced with those obstacles?"  The first question is more likely to engender "clean" answers.  The second is a request to scientificize messiness.
Aaron Sterling
CC BY-SA 2.5
2010-08-25T02:18:41.717Z
2010-08-25T02:18:41.717Z
Aaron Sterling
<p>The field of distributed computing has fallen woefully short in developing a single mathematical theory to describe distributed algorithms. There are several 'models' and frameworks of distributed computation that are simply not compatible with each other. The sheer explosion of varying temporal properties (asynchrony, synchrony, partial synchrony), various communication primitives (message passing vs. shared memory, broadcast vs. unicast), multiple fault models (fail stop, crash recover, send omission, byzantine, and so on) has left us with an intractable number of system models, frameworks, and methodologies, that comparing relative solvability results and lower bounds across these models and frameworks has become arduous, intractable, and at times, impossible.</p>
<p>My question is very simply, why is that so? What is so fundamentally different about distributed computing (from its sequential counterpart) that we haven't been able to collate the research into a unified theory of distributed computing? With sequential computing, Turing Machines, Recursive Functions, and Lambda Calculus all truned out to be equivalent. Was this just a stroke of luck, or did we really do a good job in encapsulating sequential computing in a manner that is yet to be accomplished with distributed computing?</p>
<p>In other words, is distributed computing inherently unyielding to an elegant theory (and if so, how and why?), or are we simply not smart enough to discover such a theory?</p>
<p>The only reference I could find that addresses this issue is: &quot;<a href="https://link.springer.com/article/10.1007/s00446-003-0096-6?from=SL" rel="nofollow noreferrer">Appraising two decades of distributed computing theory research</a>&quot; by Fischer and Merritt DOI: 10.1007/s00446-003-0096-6</p>
<p>Any references or expositions would be really helpful.</p>

<big-picture><dc.distributed-comp>
Why have we not been able to develop a unified complexity theory of distributed computing?
Srikanth Sastry
# Why have we not been able to develop a unified complexity theory of distributed computing?

(Asked by: username_0 on 2010-08-23)

The field of distributed computing has fallen woefully short in developing a single mathematical theory to describe distributed algorithms. There are several 'models' and frameworks of distributed computation that are simply not compatible with each other. The sheer explosion of varying temporal properties (asynchrony, synchrony, partial synchrony), various communication primitives (message passing vs. shared memory, broadcast vs. unicast), multiple fault models (fail stop, crash recover, send omission, byzantine, and so on) has left us with an intractable number of system models, frameworks, and methodologies, that comparing relative solvability results and lower bounds across these models and frameworks has become arduous, intractable, and at times, impossible.

My question is very simply, why is that so? What is so fundamentally different about distributed computing (from its sequential counterpart) that we haven't been able to collate the research into a unified theory of distributed computing? With sequential computing, Turing Machines, Recursive Functions, and Lambda Calculus all truned out to be equivalent. Was this just a stroke of luck, or did we really do a good job in encapsulating sequential computing in a manner that is yet to be accomplished with distributed computing?

In other words, is distributed computing inherently unyielding to an elegant theory (and if so, how and why?), or are we simply not smart enough to discover such a theory?

The only reference I could find that addresses this issue is: "[Appraising two decades of distributed computing theory research](https://link.springer.com/article/10.1007/s00446-003-0096-6?from=SL)" by Fischer and Merritt DOI: 10.1007/s00446-003-0096-6

Any references or expositions would be really helpful.

**Tags:** big-picture, dc.distributed-comp


---

## 4 answers

### Answer by username_5 on 2010-08-25. Score: 27 | Accepted answer

My take is that the abstractly-motivated Turing machine model of computation was a good approximation of technology until very recently, whereas models of distributed computing, from the get-go, have been motivated by the real world, which is always messier than abstractions.

From, say, 1940-1995, the size of problem instances, the relative "unimportance" of parallelism and concurrency, and the macro-scale of computing devices, all "conspired" to keep Turing machines an excellent approximation of real-world computers. However, once you start dealing with massive datasets, ubiquitous need for concurrency, biology through the algorithmic lens, etc., it is much less clear if there is an "intuitive" model of computation. Perhaps problems hard in one model are not hard -- strictly less computationally complex -- in another. So I believe that mainstream computational complexity is finally catching up (!) with distributed computing, by starting to consider multiple models of computation and data structures, motivated by real-world considerations.

> **Comments:**
> **username_5**: Also consider the defining questions of the respective fields.  "Assume you can compute perfectly.  What are the limits of what you can and can't do?" vs. "Assume you have a faulty channel, processor, or assume you have an adversary.  How can you compute successfully when faced with those obstacles?"  The first question is more likely to engender "clean" answers.  The second is a request to scientificize messiness.  

---

### Answer by username_1 on 2010-08-23. Score: 25

I will answer this from the perspective of classical graph problems (or input/output problems): we have a network, each node gets something as input and each node must produce something as output. I guess this is closest to the world of traditional computational complexity.

I am certainly biased, but I think that in this setting, there *is* a simple and fairly commonly used model of distributed computing: *synchronous distributed algorithms*, with the definition that *running time = number of synchronous rounds*. In Peleg's terminology, this is the *LOCAL* model.

This model is nice as it has very few "moving parts", no parameters, etc. Nevertheless, it is very concrete: it makes sense to say that the running time of an algorithm is exactly 15 in this model. And you can prove unconditional, information-theoretic lower bounds: from this perspective, the distributed complexity of many graph problems (e.g., graph colouring) is fairly well-understood.

This model also provides a unified approach to many aspects of distributed computing:

* Message-passing vs. shared memory, broadcast vs. unicast: Irrelevant in this model.
* Your real-world system is asynchronous? No problem, just plug in the $\alpha$-synchroniser. The time complexity (with suitable definitions) is essentially unaffected.
* You'd like to have an algorithm for dynamic networks, or you'd like to recover from failures? Well, if your synchronous algorithm is deterministic, then you can use it to construct a *self-stabilising* algorithm. Again, the time complexity is essentially unaffected.

---

Now all this is fine as long as you study problems that are "truly distributed" in the sense that the running time of your algorithm is *smaller than the diameter of the graph*, i.e., no node needs to have full information on the structure of the graph. However, there are also many problems that are inherently global: the fastest algorithm in this model has running time that is linear in the diameter of the graph. In the study of those problems, the above model no longer makes any sense, and then we need to resort to something else. Typically, one starts to pay attention to the total number of messages or bits communicated in the network. That's one reason why we get several different models.

---

Then of course we have the issue that the distributed computing community is actually [two different communities, with surprisingly few things in common](http://dx.doi.org/10.1145/1835698.1835700). If you lump together all models from two communities, it *will* certainly look a bit confusing... My answer above is related to only one half of the community; I trust others will fill in regarding the other half.

> **Comments:**
> **username_0**: If I understand this correctly, the point is that there is an elegant theory only for synchronous systems and not much else. With respect to systems other than synchronous ones, we are conflating problems/foci from two otherwise different communities, and this presents methodological issues with developing an single theory. Have I understood your arguments correctly?  
> **username_2**: Thanks for the very informative answer. I would accept this as THE answer.  

---

### Answer by username_3 on 2010-11-11. Score: 5

One romantic idea to capture various models of distributed computing has been through algebraic topology. The core idea is to construct simplicial complexes by letting points be process states, each labeled with a process id. This is a [primer](http://www.springerlink.com/content/56712l0152526304/) on the topic. The closest answer to your question has probably been touched upon by Eli gafni in his paper- Distributed computing- A glimmer of a theory. In his paper, he shows simulations how beginning with async shared memory for two-three processors(for fail stop and Byzantine)-he shows how can apply this to the message passing model. Crucial to understanding his simulations is the notion of viewing a distributed computing topologically


---

### Answer by username_4 on 2010-08-23. Score: 4

I think the situation looks quite different if viewed in context: starting from the early works and impossibility results on Byzantine agreement ([PSL80](http://portal.acm.org/citation.cfm?doid=322186.322188) [LSP82](http://portal.acm.org/citation.cfm?id=357172.357176) [FLP85](http://portal.acm.org/citation.cfm?id=214121)), it was clear soon that fundamental problems in distributed computing can only be solved at all with strict synchrony assumptions and a high degree of redundancy. As these unconditional theoretical resource lower bounds were considered infeasible for any practical purposes, research focused on developing more refined models that allowed evermore fine-grained trade-offs of assumptions (on timing guarantees or failure modes for example) vs. guarantees (i.e. number of simultaneous faults of what kinds on what type of components tolerated, e.g. processors, links) in order to give the system designers the tools to find the right trade-off for the system at hand.

> **Comments:**
> **username_0**: I understand that the refined models were introduced to understand 'practical' solvability of problems in the distributed space. One would expect these fine-grained models to arrange themselves neatly into a hierarchy with respect to solvability, time complexity, and message complexity. Unfortunately, this is not the case. My question here, is what is the reason for this balkanization? If it is some attribute(s) inherent to distributed computing, then what are they?  

---

