<p>The ideas the question expresses are interesting but maybe insufficiently
fleshed out.  I can see a couple of points that deserve further
refinement.</p>

<ul>
<li><p>It is difficult to "code the exact same functionality". In part that
is because what counts as "the exact same functionality" depends on
the chosen notion of program equivalence. For terminating programs
in sequential programming languages there's a canonical definition,
but when you move to concurrent, non-terminating programs,
canonicity vanishes, and you are left with multiple, reasonable
choices that are mutually incompatible. Secondly, but related to the
first point, different programming languages are fundamentally
different: things you can do in a low-level language can often not
be done in a high-level language. A typical example is to do with
speed.  Certain tasks can be solved intrinsically faster in assembly language
than in e.g. Prolog or Javascript (assuming conventional
compilation).</p></li>
<li><p>The KQuotient is subjective, in the sense that for any
given task, programmers A and B will likely have different length
base and test language implementations.</p></li>
<li><p>Moreover, the KQuotient is also task dependent, meaning that, given
a pair of languages, the KQuotient for Task A will in general be
different from the KQuotient of Task B.</p></li>
</ul>

<p>Finally, even though I find the idea of measuring maintainability via
query numbers to reach consensus intriguing, I suggest to be more
clear about what kind of queries are relevant here (syntax?
semantics?), and how the concept of programmer consensus can be made
precise.</p>

CC BY-SA 3.0
2013-06-21T03:19:34.097Z
@MarkJ: I like your ideas! But a word of caution. Even in a sequential synchronous fashion where output can arrive predictably, defining "functionality" takes some care. Think about implementing, say, mergesort vs bubblesort. They have the same input-output behavior but look very different even in the same language. If you want an example where they both have similar resource usage, then compare mergesort and quicksort (or heapsort or ...). To get around M. Berger's point about different programmers, you could consider an average over programmers with "sufficient experience."
Joshua Grochow
CC BY-SA 3.0
2013-06-21T09:26:54.647Z
@JoshuaGrochow How does considering an average over programmers with "sufficient experience" avoid subjectivity? In order to formalise query complexity of programs, or related concepts, one would have to rely on syntactic features of programs only.
Martin Berger
CC BY-SA 3.0
2013-06-21T14:59:46.367Z
@MartinBerger: I didn't mean to say it completely *avoids* subjectivity, just that it helps reduce it. I agree that the notions of expressiveness and maintainability are subjective - perhaps even necessarily so. So completely avoiding subjectivity probably shouldn't be a goal. But one still wants to avoid the subjective bias of any *single* programmer.
Joshua Grochow
CC BY-SA 3.0
2013-06-21T15:26:54.240Z
@JoshuaGrochow I agree that practically relevant notions of expressiveness are strongly related with programmer knowledge, hence subjective. However, I think some of the ideas on programming language expressiveness (via compression and 'maintainability') suggested by Mark might be formalisable, and have interesting formal properties.
Martin Berger
CC BY-SA 3.0
2013-06-11T19:08:46.620Z
I was speaking of "functionality" in a non-parallel, synchronous fashion where output can arrive predictably.  I will have to make that clearer, because I don't believe the concept can be extended to multi-threaded programs running concurrently.  On the second point, I addressed this somewhat by the concept of "maintainability".
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2014-05-26T19:54:12.047Z
@MartinBerger I've added a bit to the definition to show how it avoids subjectivity by noting that most programmers *can* actually agree to whether one implementation is better over another (when this isn't the case, it remains in a distinctly finite set of "equivalent programs" whose size doesn't ever seem to get greater than 2).
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2014-07-20T20:51:39.420Z
@MartinBerger: edit2, I've now incorporated "equivalent programs" directly into the definition and defined exactly what I mean (even if limited from other's pov).
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2013-06-10T07:37:22.813Z
2013-06-10T09:42:19.963Z
2013-06-10T09:42:19.963Z
Martin Berger
<p>here is a project <a href="http://rosettacode.org/wiki/Rosetta_Code" rel="noreferrer">Rosetta Code</a> somewhat similar or adaptable to some of your goals [which as others point out are not very specific/objective/concise/clearcut yet], a database involving quantification of languages on the same task for comparison. the post <a href="http://blog.wolfram.com/2012/11/14/code-length-measured-in-14-languages/" rel="noreferrer">"Code Length Measured in 14 Languages"</a> is an example of the quantitative analysis possible with this database, mostly focused on Mathematica versus other languages. </p>

<blockquote>
  <p>Rosetta Code is a programming chrestomathy site. The idea is to present solutions to the same task in as many different languages as possible, to demonstrate how languages are similar and different, and to aid a person with a grounding in one approach to a problem in learning another. Rosetta Code currently has 676 tasks, 102 draft tasks, and is aware of 503 languages, though we do not (and cannot) have solutions to every task in every language.</p>
</blockquote>

CC BY-SA 3.0
2013-09-27T23:00:23.137Z
Nice, will review.
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2013-06-14T21:19:08.613Z
2013-06-14T21:25:16.593Z
2013-06-14T21:25:16.593Z
vzn
<p>the <a href="http://rosettacode.org/wiki/Rosetta_Code" rel="nofollow">Rosetta Code</a> repository has been used for a new comprehensive scientific/ academic study of language succinctness among other comparisons of properties. announced at <a href="http://se.inf.ethz.ch/people/nanz/research/rosettacode.html" rel="nofollow">"Analyzing Programming Languages using Rosetta Code"</a></p>

<ul>
<li><a href="http://arxiv.org/abs/1409.0252" rel="nofollow">A Comparative Study of Programming Languages in Rosetta Code</a> Nanz/ Furia</li>
</ul>

<blockquote>
  <p>Sometimes debates on programming languages are more religious than scientific. Questions about which language is more succinct or efficient, or makes developers more productive are discussed with fervor, and their answers are too often based on anecdotes and unsubstantiated beliefs. In this study, we use the largely untapped research potential of Rosetta Code, a code repository of solutions to common programming tasks in various languages, to draw a fair and well-founded comparison. Rosetta Code offers a large data set for analysis. Our study is based on 7087 solution programs corresponding to 745 tasks in 8 widely used languages representing the major programming paradigms (procedural: C and Go; object-oriented: C# and Java; functional: F# and Haskell; scripting: Python and Ruby). Our statistical analysis reveals, most notably, that: functional and scripting languages are more concise than procedural and object-oriented languages; C is hard to beat when it comes to raw speed on large inputs, but performance differences over inputs of moderate size are less pronounced and allow even interpreted languages to be competitive; compiled strongly-typed languages, where more defects can be caught at compile time, are less prone to runtime failures than interpreted or weakly-typed languages. We discuss implications of these results for developers, language designers, and educators,</p>
</blockquote>

CC BY-SA 3.0
2014-09-24T17:42:13.807Z
2014-09-24T17:42:13.807Z
vzn
<p>I want to explore the notion of quantifying the amount of <em>succinctness</em> a programming language provides.  That is, the amount a high-level language reduces the complex. </p>

<p>This idea of "simplification" is a factor of <strong>text-wise reduction</strong> (fewer characters needed to express a complex concept, à la Algorithmic Information Theory) and another, less easy-to-quantify concept of <strong>maintainability</strong>. Fleshing out this latter concept, it is clear it has to do with how easily one can establish programmer consensus for the given task (i.e. how many programmers of the language would put it back the same way you've expressed it or otherwise agree on the best implementation between different implementations of the same problem?).</p>

<p>I will define the "Kolmogorov Quotient" so that higher numbers for a given language denote a <em>reduction</em> in the complexity of solving the problem in the given language.</p>

<p>The metric for "text-wise reduction" should incorporate a constant factor based on (non-identifier) symbols used in the language and source text.  These factors will be the same across all languages implemented (i.e. <em>designed</em>) for a given architecture (e.g. VonNeumann architecture vs. Symbolics) and will be a measure of the <strong><em>significance</em></strong> of the symbol; i.e. the topology of the language tokens.  (These terms, alas, will also need to be fleshed out and defined.)  </p>

<p>Once the basic premise and a methodology above is agreed to, it is only a matter of a rough constant of difference for any specific implementation/architecture. (That is, as long as the architecture is the same across all measurements, the number should be valid and comparable between languages.) </p>

<p>But it could go something like this: Pick a language "close to the machine", like C or Assembly, and measure the amount of bytes of machine code it used to implement a standard suite(*) of common, non-threaded programming tasks (base_language_count). Then code the exact same functionality in the language you are wanting to measure (without using external libraries) and count the number of bytes of source code (test_language_count).</p>

<p>KQuotient = (base_language_count / test_language_count) / number_of_equal_programs. </p>

<p>"number_of_equal_programs" is the number of programs fluent programmers of the language agree that are the best and equal solutions to the problem. (I will define "equal programs" as those who's output is the same for every input.) </p>

<p>The first ratio <em>should</em> always be greater than 1.0.  My only concern is that for each programming language, one could reduce the KQuotient number simply by changing each keyword to a single character.</p>

<p>(*) "standard suite of common programming tasks...":  I see two main categories:</p>

<ol>
<li>Data-processing suite, limited to simple text I/O <em>(computation towards the machine)</em></li>
<li>GUI suite <em>(computation towards the user)</em></li>
</ol>

<p><em>The purpose of this idea is to end the tiring "language wars" about whose language is the best.  By giving a quantitative metric, people can at least argue better.</em></p>

CC BY-SA 3.0
2013-06-14T21:12:23.817Z
what you define as reduction vs maintainability are arguably two sometimes orthogonal/not-fully-overlapping concepts. however, have noticed also how languages tend to build on each other in a sort of pyramidal structure, which nobody seems to have studied or quantified much.
vzn
CC BY-SA 3.0
2013-06-21T03:24:53.167Z
@MarkJ: The definition of "external libraries" seems tricky to me. For example, lists are essentially built in to languages like scheme or haskell, but are a library for C (assembly, etc.), and are part of the "standard library" (not usually considered external) for Java. Maybe one could consider implementations of a given algorithm given a black box for some data structure (like lists etc.) In a similar vein, for any function f I could define a (admittedly unnatural) language which had f as a basic primitive, hence would not be external and would have a *very* short program in that language.
Joshua Grochow
CC BY-SA 3.0
2013-09-11T02:14:48.683Z
@JoshuaGrochow:  Yes, yet it says something when a feature is bundled with a language (albeit in its included library) vs. a library maintained by 3rd parties and not included in the "canonical distribution".  One is maintained along with the language, the other is not.
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2013-09-11T04:36:28.230Z
a few more thoughts on this. (1) programmers will almost never voluntary write large amts of logically equivalent code in two different ways (2) there is a case where it happens, large project porting/migration. then LOC (lines of code) can be compared in each platform pre/post migration. another case where it happens is eg games written on different gaming platforms. another case is with the GUI where there are competing front-end standards to do the same thing eg HTML5/js/web2.0 vs flash/swing etc.... there one could look at open source projects that were ported...
vzn
CC BY-SA 3.0
2013-10-28T10:06:09.720Z
I'm not sure the bytes is what you want to consider. What about considering terminal symbols and their entropy, and then grammatical (syntactical) constructions and their entropy nesting them up to the size of the program? A succinct language should have high entropy in the symbols, sparing the programmer from repetitions, low entropy in the grammar would mean it's a simple to learn language. Fewer grammar constructions in the sum of a program means more powerful ones, good fun. In short, I think you should compare **trees**, not characters and not *exactly* algorithms (Kolmogorov). LMKWYT.
Trylks
CC BY-SA 3.0
2013-06-10T04:10:56.900Z
I don't see any question in your post.
Kaveh
CC BY-SA 3.0
2013-06-10T11:46:56.737Z
You might want to explore another name for what you're looking at.  Expressiveness has been previously discussed PL research. See Fellesien's "On the Expressive Power of Programming Languages".
Logan Mayfield
CC BY-SA 3.0
2013-06-10T12:27:20.457Z
@MartinBerger I mentioned Felleisen's paper not because it solves his problem, but because if he pursues his idea he might want to find another name for what he's measuring or at least something to differentiate it from Felleisen's expressiveness.
Logan Mayfield
CC BY-SA 3.0
2014-05-26T23:53:28.393Z
@vzn: the orthogonality between text-wise reduction and maintainability is intentional.  By maximizing independence from each other, one gleans more information from the number.
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2014-07-20T20:44:50.317Z
@Trylks:  I like the symbol entropy idea, and I've incorporated that a bit in my text.  As for comparing [lexical] trees, that would miss the point, because one significant part of this is to *maintain the **constraints*** of the underlying hardware architecture, otherwise without the tension between hardware and software, the metric really measures nothing.  (This is, by the way, against the orthodoxy which says "all Turing-complete languages are equal", but I contest that thesis on a practical level).
Mark Rosenblitt-Janssen
CC BY-SA 3.0
2014-09-18T19:50:20.960Z
I'd suggesting including in the "data-processing suite" something like a collections library (such as those for C++ or Java). Simple I/O alone seems *too* simple, in addition to being more architecture-dependent.
Joshua Grochow
<pl.programming-languages><kolmogorov-complexity><algebraic-topology>
How to measure programming language succinctness?
Mark Rosenblitt-Janssen
# How to measure programming language succinctness?

(Asked by: username_0 on 2013-06-10)

I want to explore the notion of quantifying the amount of *succinctness* a programming language provides. That is, the amount a high-level language reduces the complex.

This idea of "simplification" is a factor of **text-wise reduction** (fewer characters needed to express a complex concept, à la Algorithmic Information Theory) and another, less easy-to-quantify concept of **maintainability**. Fleshing out this latter concept, it is clear it has to do with how easily one can establish programmer consensus for the given task (i.e. how many programmers of the language would put it back the same way you've expressed it or otherwise agree on the best implementation between different implementations of the same problem?).

I will define the "Kolmogorov Quotient" so that higher numbers for a given language denote a *reduction* in the complexity of solving the problem in the given language.

The metric for "text-wise reduction" should incorporate a constant factor based on (non-identifier) symbols used in the language and source text. These factors will be the same across all languages implemented (i.e. *designed*) for a given architecture (e.g. VonNeumann architecture vs. Symbolics) and will be a measure of the ***significance*** of the symbol; i.e. the topology of the language tokens. (These terms, alas, will also need to be fleshed out and defined.)

Once the basic premise and a methodology above is agreed to, it is only a matter of a rough constant of difference for any specific implementation/architecture. (That is, as long as the architecture is the same across all measurements, the number should be valid and comparable between languages.)

But it could go something like this: Pick a language "close to the machine", like C or Assembly, and measure the amount of bytes of machine code it used to implement a standard suite(\*) of common, non-threaded programming tasks (base\_language\_count). Then code the exact same functionality in the language you are wanting to measure (without using external libraries) and count the number of bytes of source code (test\_language\_count).

KQuotient = (base\_language\_count / test\_language\_count) / number\_of\_equal\_programs.

"number\_of\_equal\_programs" is the number of programs fluent programmers of the language agree that are the best and equal solutions to the problem. (I will define "equal programs" as those who's output is the same for every input.)

The first ratio *should* always be greater than 1.0. My only concern is that for each programming language, one could reduce the KQuotient number simply by changing each keyword to a single character.

(\*) "standard suite of common programming tasks...": I see two main categories:

1. Data-processing suite, limited to simple text I/O *(computation towards the machine)*
2. GUI suite *(computation towards the user)*

*The purpose of this idea is to end the tiring "language wars" about whose language is the best. By giving a quantitative metric, people can at least argue better.*

**Tags:** pl.programming-languages, kolmogorov-complexity, algebraic-topology
> **Comments:**
> **username_4**: I don't see any question in your post.  
> **username_5**: You might want to explore another name for what you're looking at.  Expressiveness has been previously discussed PL research. See Fellesien's "On the Expressive Power of Programming Languages".  
> **username_5**: @MartinBerger I mentioned Felleisen's paper not because it solves his problem, but because if he pursues his idea he might want to find another name for what he's measuring or at least something to differentiate it from Felleisen's expressiveness.  
> **username_1**: what you define as reduction vs maintainability are arguably two sometimes orthogonal/not-fully-overlapping concepts. however, have noticed also how languages tend to build on each other in a sort of pyramidal structure, which nobody seems to have studied or quantified much.  
> **username_2**: @MarkJ: The definition of "external libraries" seems tricky to me. For example, lists are essentially built in to languages like scheme or haskell, but are a library for C (assembly, etc.), and are part of the "standard library" (not usually considered external) for Java. Maybe one could consider implementations of a given algorithm given a black box for some data structure (like lists etc.) In a similar vein, for any function f I could define a (admittedly unnatural) language which had f as a basic primitive, hence would not be external and would have a *very* short program in that language.  
> **username_0**: @JoshuaGrochow:  Yes, yet it says something when a feature is bundled with a language (albeit in its included library) vs. a library maintained by 3rd parties and not included in the "canonical distribution".  One is maintained along with the language, the other is not.  
> **username_1**: a few more thoughts on this. (1) programmers will almost never voluntary write large amts of logically equivalent code in two different ways (2) there is a case where it happens, large project porting/migration. then LOC (lines of code) can be compared in each platform pre/post migration. another case where it happens is eg games written on different gaming platforms. another case is with the GUI where there are competing front-end standards to do the same thing eg HTML5/js/web2.0 vs flash/swing etc.... there one could look at open source projects that were ported...  
> **username_3**: I'm not sure the bytes is what you want to consider. What about considering terminal symbols and their entropy, and then grammatical (syntactical) constructions and their entropy nesting them up to the size of the program? A succinct language should have high entropy in the symbols, sparing the programmer from repetitions, low entropy in the grammar would mean it's a simple to learn language. Fewer grammar constructions in the sum of a program means more powerful ones, good fun. In short, I think you should compare **trees**, not characters and not *exactly* algorithms (Kolmogorov). LMKWYT.  
> **username_0**: @vzn: the orthogonality between text-wise reduction and maintainability is intentional.  By maximizing independence from each other, one gleans more information from the number.  
> **username_0**: @Trylks:  I like the symbol entropy idea, and I've incorporated that a bit in my text.  As for comparing [lexical] trees, that would miss the point, because one significant part of this is to *maintain the **constraints*** of the underlying hardware architecture, otherwise without the tension between hardware and software, the metric really measures nothing.  (This is, by the way, against the orthodoxy which says "all Turing-complete languages are equal", but I contest that thesis on a practical level).  
> **username_2**: I'd suggesting including in the "data-processing suite" something like a collections library (such as those for C++ or Java). Simple I/O alone seems *too* simple, in addition to being more architecture-dependent.  

---

## 3 answers

### Answer by username_6 on 2013-06-10. Score: 7

The ideas the question expresses are interesting but maybe insufficiently
fleshed out. I can see a couple of points that deserve further
refinement.

* It is difficult to "code the exact same functionality". In part that
  is because what counts as "the exact same functionality" depends on
  the chosen notion of program equivalence. For terminating programs
  in sequential programming languages there's a canonical definition,
  but when you move to concurrent, non-terminating programs,
  canonicity vanishes, and you are left with multiple, reasonable
  choices that are mutually incompatible. Secondly, but related to the
  first point, different programming languages are fundamentally
  different: things you can do in a low-level language can often not
  be done in a high-level language. A typical example is to do with
  speed. Certain tasks can be solved intrinsically faster in assembly language
  than in e.g. Prolog or Javascript (assuming conventional
  compilation).
* The KQuotient is subjective, in the sense that for any
  given task, programmers A and B will likely have different length
  base and test language implementations.
* Moreover, the KQuotient is also task dependent, meaning that, given
  a pair of languages, the KQuotient for Task A will in general be
  different from the KQuotient of Task B.

Finally, even though I find the idea of measuring maintainability via
query numbers to reach consensus intriguing, I suggest to be more
clear about what kind of queries are relevant here (syntax?
semantics?), and how the concept of programmer consensus can be made
precise.

> **Comments:**
> **username_0**: I was speaking of "functionality" in a non-parallel, synchronous fashion where output can arrive predictably.  I will have to make that clearer, because I don't believe the concept can be extended to multi-threaded programs running concurrently.  On the second point, I addressed this somewhat by the concept of "maintainability".  
> **username_2**: @MarkJ: I like your ideas! But a word of caution. Even in a sequential synchronous fashion where output can arrive predictably, defining "functionality" takes some care. Think about implementing, say, mergesort vs bubblesort. They have the same input-output behavior but look very different even in the same language. If you want an example where they both have similar resource usage, then compare mergesort and quicksort (or heapsort or ...). To get around M. Berger's point about different programmers, you could consider an average over programmers with "sufficient experience."  
> **username_6**: @JoshuaGrochow How does considering an average over programmers with "sufficient experience" avoid subjectivity? In order to formalise query complexity of programs, or related concepts, one would have to rely on syntactic features of programs only.  
> **username_2**: @MartinBerger: I didn't mean to say it completely *avoids* subjectivity, just that it helps reduce it. I agree that the notions of expressiveness and maintainability are subjective - perhaps even necessarily so. So completely avoiding subjectivity probably shouldn't be a goal. But one still wants to avoid the subjective bias of any *single* programmer.  
> **username_6**: @JoshuaGrochow I agree that practically relevant notions of expressiveness are strongly related with programmer knowledge, hence subjective. However, I think some of the ideas on programming language expressiveness (via compression and 'maintainability') suggested by Mark might be formalisable, and have interesting formal properties.  
> **username_0**: @MartinBerger I've added a bit to the definition to show how it avoids subjectivity by noting that most programmers *can* actually agree to whether one implementation is better over another (when this isn't the case, it remains in a distinctly finite set of "equivalent programs" whose size doesn't ever seem to get greater than 2).  
> **username_0**: @MartinBerger: edit2, I've now incorporated "equivalent programs" directly into the definition and defined exactly what I mean (even if limited from other's pov).  

---

### Answer by username_1 on 2013-06-14. Score: 4

here is a project [Rosetta Code](http://rosettacode.org/wiki/Rosetta_Code) somewhat similar or adaptable to some of your goals [which as others point out are not very specific/objective/concise/clearcut yet], a database involving quantification of languages on the same task for comparison. the post ["Code Length Measured in 14 Languages"](http://blog.wolfram.com/2012/11/14/code-length-measured-in-14-languages/) is an example of the quantitative analysis possible with this database, mostly focused on Mathematica versus other languages.

> Rosetta Code is a programming chrestomathy site. The idea is to present solutions to the same task in as many different languages as possible, to demonstrate how languages are similar and different, and to aid a person with a grounding in one approach to a problem in learning another. Rosetta Code currently has 676 tasks, 102 draft tasks, and is aware of 503 languages, though we do not (and cannot) have solutions to every task in every language.

> **Comments:**
> **username_0**: Nice, will review.  

---

### Answer by username_1 on 2014-09-24. Score: 1

the [Rosetta Code](http://rosettacode.org/wiki/Rosetta_Code) repository has been used for a new comprehensive scientific/ academic study of language succinctness among other comparisons of properties. announced at ["Analyzing Programming Languages using Rosetta Code"](http://se.inf.ethz.ch/people/nanz/research/rosettacode.html)

* [A Comparative Study of Programming Languages in Rosetta Code](http://arxiv.org/abs/1409.0252) Nanz/ Furia

> Sometimes debates on programming languages are more religious than scientific. Questions about which language is more succinct or efficient, or makes developers more productive are discussed with fervor, and their answers are too often based on anecdotes and unsubstantiated beliefs. In this study, we use the largely untapped research potential of Rosetta Code, a code repository of solutions to common programming tasks in various languages, to draw a fair and well-founded comparison. Rosetta Code offers a large data set for analysis. Our study is based on 7087 solution programs corresponding to 745 tasks in 8 widely used languages representing the major programming paradigms (procedural: C and Go; object-oriented: C# and Java; functional: F# and Haskell; scripting: Python and Ruby). Our statistical analysis reveals, most notably, that: functional and scripting languages are more concise than procedural and object-oriented languages; C is hard to beat when it comes to raw speed on large inputs, but performance differences over inputs of moderate size are less pronounced and allow even interpreted languages to be competitive; compiled strongly-typed languages, where more defects can be caught at compile time, are less prone to runtime failures than interpreted or weakly-typed languages. We discuss implications of these results for developers, language designers, and educators,


---

