<p>Matthias Felleisen presented a compelling solution to the side effects problem in semantics in his series on "Syntactic Theories of Control and State."</p>

<p>That line of work resulted in the CESK machine, a simple abstract machine framework capable of concisely modeling functional, object-oriented, imperative and even logic languages.  The CESK framework handles not just side effects, but also "complex" control constructs like exceptions, continuations, laziness and even threads.</p>

<p>The CESK machine, and small-step operational semantics more broadly, have been the de facto standard in programming language theory for about two decades.</p>

<p>In short, a CESK machine is a small-step machine with four components to describe every machine state: the control string (a generalization of the program counter), the environment, a store (also called the heap) and the current continuation.</p>

<p>The environment maps variables to addresses; the store maps addresses to values.</p>

<p>This makes it straightforward to model mutable variables: just change the value at its address.</p>

<p>It also makes it easy to model pointers and dynamic allocation: just make store addresses first-class values.</p>

<p>In a similar fashion, first-class continuations result from making them addressable values.</p>

CC BY-SA 2.5
2010-08-29T04:50:54.367Z
2010-08-29T04:50:54.367Z
Matt Might
<p>There are of course ways of dealing with effects in (denotational) semantics. For example, we can use <a href="http://www.disi.unige.it/person/MoggiE/">Eugenio Moggi's</a> idea that <a href="http://www.disi.unige.it/person/MoggiE/ftp/ic91.pdf">computational effects are monads</a> (this idea has also been used in the design of Haskell). One of the problems with this is that monads are hard to combine. <a href="http://homepages.inf.ed.ac.uk/gdp/">Gordon Plotkin</a> and <a href="http://www.bath.ac.uk/comp-sci/people/contact/index.php?contact=Dr_John_Power">John Power</a> suggested <a href="http://homepages.inf.ed.ac.uk/gdp/publications/Overview.pdf">a refinement of Moggi's monads to Lawvere theories</a>, or algebraic theories as they are also called, which encompasses <em>algebraic</em> effects (most common effects are algebraic, such as state, I/O, non-determinism, but continuations are not). For a comprehensive treatment, see <a href="http://matija.pretnar.info/handlers-of-algebraic-effects.pdf">Matija Pretnar's thesis</a>.</p>

<p>I should also mention the <a href="http://portal.acm.org/citation.cfm?id=910747">possible worlds semantics</a> for local state, developed by Frank Oles and <a href="http://www.cs.cmu.edu/~jcr/">John Reynolds</a> (sorry, can't find a better link, this stuff is from 1982), which predates Moggi's monads. They used categories of presheaves to provide a semantics of an algol-like language which correctly modeled many aspects of local state (but not all of them, I think the model allowed snapback, but maybe my memory serves me wrong).</p>

CC BY-SA 3.0
2012-02-28T21:38:57.850Z
O'Hearn and Tennent model is not fully abstract.  That is discussed in the paper itself.  But the refinement by O'Hearn and Reynolds using linear lambda calculus is fully abstract up to second-order.  It breaks for third-order, examples being the equivalences studied by Ahmed, Dreyer, Birkedal et al.
Uday Reddy
CC BY-SA 2.5
2010-08-29T09:32:23.800Z
Yes, the functor category semantics didn't validate all of the Meyer-Sieber equivalences. Peter O'Hearn and Robert Tennant developed a parametric version of the functor-category semantics in the mid-90s which (IIRC) got all of the Meyer-Sieber examples, but I don't know if it was fully abstract or not.
Neel Krishnaswami
CC BY-SA 2.5
2010-08-28T21:44:42.737Z
2010-08-28T23:19:04.830Z
2010-08-28T23:19:04.830Z
Andrej Bauer
<p><em>How does the existence of side effects in a programming language impact the ability to map a program to a computational model?</em></p>

<p>It doesn't necessarily make it difficult, but it does impose restrictions on the way the semantics of larger expressions can be constructed out of smaller ones.  It can interact very badly with certain other programming constructs, for instance, if one wants to give a Scott-style denotational semantics for a language allowing assignment of higher-order functions to global references.</p>

<p>It is not simply side effects like state that cause the trouble.  Simple imperative languages such as Dijkstra's guarded command language have these kinds of side effects, and have nice semantics.  Trouble arises with extensions of the lambda-calculus with the kind of operational semantics expected of programming languages even in the absence of side effects: the earliest, Plotkin's PCF, was given denotational models relatively early, but the semantics were not fully abstract, meaning that the denotational semantics was overly general, not exactly corresponding to their operational semantics.  PCF finally recieved a fully abstract denotational semantics in the late 1980s with game semantics, which is not at all like Scott's order-theoretic semantics.  Concurrency still has not received a fully adequate denotational treatment.  </p>

<p>Many question the importance of this kind of semantics.  We can always provide some kind of operational semantics, even if that "semantics" is just the program source and the names of some machines that have compiled and run the program: for this reason Strachey condemned operational semantics.  But Plotkin's structural operation semantics has shown how operational semantics can be separated from machine models, and Pitt's work has shown how such semantics can support similar reasoning about programs and programming languages to denotational semantics.  Thus operational semantics is a viable alternative to denotational semantics, and have been applied with success to a substantial number of programming languages such as Standard ML.</p>

<p><em>Are there approaches to managing state that can improve this process while still allowing side effects?</em></p>

<p>To some extent, difficulties providing semantics correspond to the difficulty of providing powerful programming languages that behave in the way one would expect.  Pragmatically motivated design decisions &mdash;such as avoiding the use of global state together with concurrency, typically through message-passing concurrency&mdash; make it easier to provide semantics.  </p>

CC BY-SA 3.0
2012-02-28T19:26:14.447Z
But the point remains that PCF doesn't have state.  What "reason" are you saying that Strachey to condemn operational semantics?  It wasn't apparent to me.  The last paragraph contradicts what you said earlier, viz., guarded commands have nice semantics but PCF doesn't!
Uday Reddy
CC BY-SA 3.0
2013-05-16T08:08:51.653Z
@Andrej, Uday: I've fixed my post, less than three years later.
Charles Stewart
CC BY-SA 2.5
2010-08-30T18:57:53.350Z
Scott's PCF does not have state and neither is it Scott's, is it?See http://en.wikipedia.org/wiki/Programming_language_for_Computable_Functions
Andrej Bauer
CC BY-SA 2.5
2010-08-31T11:47:57.827Z
@Andrej: Err, quite, given that Luke Ong supervised my D.Phil, I should not make that mistake.  I posted a teaser-summary of Milner's PCF and Scott's LCF that is more ... succint than WP's as an LtU story: http://lambda-the-ultimate.org/node/2196  It occurs to me that you might have access to the missing Scott (1969) manuscript...
Charles Stewart
CC BY-SA 2.5
2010-08-31T13:18:32.493Z
That would be Plotkin's PCF, I think :-) I can try to get hold of the manuscript, but I don't actually have it.
Andrej Bauer
CC BY-SA 3.0
2010-08-27T09:19:02.660Z
2013-05-16T08:07:35.180Z
2013-05-16T08:07:35.180Z
Charles Stewart
<p>Building on Charles's answer, the main difficulty in the theory of programming languages is that the natural notion of equivalence of programs is typically not strict equality either in the most straightforward mathematical semantics you can give, or in the underlying machine model. For example, consider the following bit of Java-like code: </p>

<pre><code>Object x = new Object();
Object y = new Object();
... some more code ...
</code></pre>

<p>So this program creates an object and names it x, and then creates a second object named y, and then continues executing some more code. Now, suppose that a programmer decides to flip the order of allocation of these two objects:</p>

<pre><code>Object y = new Object();
Object x = new Object();
... some more code ...
</code></pre>

<p>Now, ask the question: does this refactoring change the behavior of the program? On the one hand, on the underlying machine, x and y will be allocated at different locations in the two runs of the program. So in this sense, the program behaves differently.</p>

<p>But in a Java-like language, you can only test references for equality, and not for order, so this is a difference that the "some more code" <em>cannot observe</em>.  As a result, most programmers will expect that reversing the order will make no difference to the final answer, and most compiler writers expect to be able to perform reorderings and optimizations on this basis. (On the other hand, in a C-like language, you <em>can</em> compare pointers for ordering, by casting them to integers first, and so this reordering does <em>not</em> necessarily preserve observable behavior.) </p>

<p>One of the central questions of semantics is to answer the question of when two programs are observably equivalent. Since our notion of observation depends on the features of the programming language, we end up with a definition like "two programs are equivalent when no client program can compute different answers based on receiving those programs as inputs." The quantification over all client programs is what makes this question difficult -- it seems like you end up having to say something about all possible client programs to say something about two particular pieces of code.</p>

<p>The trick with denotational semantics is to give a mathematical interpretation that lets you avoid this universal quantification -- you say that the meaning of a piece of code is some mathematical value, and you compare them by checking to see if they're mathematically equal or not. This is local (ie, compositional), and does not involve quantification over all possible clients. (You do need to show that the denotational semantics implies contextual equivalence for it to be sound, of course. When it is complete -- when denotational equality is exactly the same as contextual equivalence, we say the semantics is "fully abstract".)</p>

<p>But means that you need to ensure that the denotational semantics validates those equivalences. So for this example, if you wanted to give a denotational semantics for this Java-like language, you need to ensure not just that calling new takes a heap and gives you back a new heap with the newly-created object, but that the meaning of the program is invariant same under all permutations of the input heap. This can involve quite complex mathematical structures (eg, in this case working in a category which ensures everything works modulo a suitable permutation group). </p>

CC BY-SA 3.0
2012-04-30T17:28:39.100Z
@RaduGRIGore: program context means something else. You are reading correctly the post, but if X reads the source code of Y as input (which is how I interpret the post), you can distinguish every two syntactically different programs; instead if Y is a lambda function on X, you can distinguish too few programs.
Neel's comment about "program context" is the correct definition: a program context Y is a program with a hole in its AST, where you can place (meaningfully) two different program fragments X1 and X2.
Blaisorblade
CC BY-SA 3.0
2012-04-30T17:30:19.863Z
@NeelKrishnaswami: could you maybe clarify what you mean in the post? You can just continue using your example and talk about a program where you can insert one or the other fragment.
Blaisorblade
CC BY-SA 2.5
2010-08-27T10:20:57.233Z
"two programs are equivalent when no client program can compute different answers based on receiving those programs as inputs." I'm confused by this. If you have a program X and a *client* program Y, then I take it to mean that Y 'calls into' X. But then you seem to say that Y reads the text of X as *input*, in which case I'd hardly call Y a 'client' of X. Could you please clarify?
Radu GRIGore
CC BY-SA 2.5
2010-08-27T21:52:15.920Z
By "client of X", I just mean the same as "program context", which is just a "larger program which contains X as a subterm".
Neel Krishnaswami
CC BY-SA 2.5
2010-08-31T09:00:55.233Z
So you use 'X is a client of Y' interchangeably with 'X reads Y as input' because you think of X as a lambda applied to Y? It makes sense, but it's a little twisted when you talk about Java. :)
Radu GRIGore
CC BY-SA 2.5
2010-08-27T09:44:09.743Z
2010-08-27T09:44:09.743Z
Neel Krishnaswami
<p>In Anthony Aaby's <a href="http://www.emu.edu.tr/aelci/Courses/D-318/D-318-Files/plbook/index.htm">"Introduction to Programming Languages"</a> section on <a href="http://www.emu.edu.tr/aelci/Courses/D-318/D-318-Files/plbook/semantic.htm">Semantics</a>, he makes the following observation:</p>

<blockquote>
  <p>Much of the work in the semantics of
  programming languages is motivated by
  the problems encountered in trying to
  construct and understand imperative
  programs---programs with assignment
  commands. Since the assignment command
  reassigns values to variables, the
  assignment can have unexpected effects
  in distant portions of the program.</p>
</blockquote>

<p>This strikes me as a remarkable admission, that allowing side effects would motivate a major part of the work in semantics.  </p>

<p>How does the existence of <a href="http://en.wikipedia.org/wiki/Side_effect_%28computer_science%29">side effects</a> in a programming language impact the ability to map a program to a computational model?  Are there approaches to managing state that can improve this process while still allowing side effects?</p>

CC BY-SA 2.5
2010-08-27T10:06:13.950Z
Should this be tagged as a soft-question? Since "much of the work in semantics [...] is motivated by [side-effects]", surely you can't expect a short and rigorous answer.
Radu GRIGore
CC BY-SA 2.5
2010-08-27T10:33:00.043Z
@Radu: On MO, this would probably be tagged [big-picture], which mostly are not [soft-question] or CW there.
Charles Stewart
CC BY-SA 2.5
2010-08-27T10:51:46.170Z
The tag big-picture is even better. I forgot about it.
Radu GRIGore
CC BY-SA 2.5
2010-08-27T11:39:49.047Z
Good suggestion; I added the tag.
Shane
<imperative-programming><semantics><big-picture>
How are side effects handled in semantics?
Shane
# How are side effects handled in semantics?

(Asked by: username_0 on 2010-08-26)

In Anthony Aaby's ["Introduction to Programming Languages"](http://www.emu.edu.tr/aelci/Courses/D-318/D-318-Files/plbook/index.htm) section on [Semantics](http://www.emu.edu.tr/aelci/Courses/D-318/D-318-Files/plbook/semantic.htm), he makes the following observation:

> Much of the work in the semantics of
> programming languages is motivated by
> the problems encountered in trying to
> construct and understand imperative
> programs---programs with assignment
> commands. Since the assignment command
> reassigns values to variables, the
> assignment can have unexpected effects
> in distant portions of the program.

This strikes me as a remarkable admission, that allowing side effects would motivate a major part of the work in semantics.

How does the existence of [side effects](http://en.wikipedia.org/wiki/Side_effect_%28computer_science%29) in a programming language impact the ability to map a program to a computational model? Are there approaches to managing state that can improve this process while still allowing side effects?

**Tags:** imperative-programming, semantics, big-picture

> **Comments:**
> **username_1**: Should this be tagged as a soft-question? Since "much of the work in semantics [...] is motivated by [side-effects]", surely you can't expect a short and rigorous answer.  
> **username_2**: @Radu: On MO, this would probably be tagged [big-picture], which mostly are not [soft-question] or CW there.  
> **username_1**: The tag big-picture is even better. I forgot about it.  
> **username_0**: Good suggestion; I added the tag.  

---

## 4 answers

### Answer by username_6 on 2010-08-27. Score: 18 | Accepted answer

Building on Charles's answer, the main difficulty in the theory of programming languages is that the natural notion of equivalence of programs is typically not strict equality either in the most straightforward mathematical semantics you can give, or in the underlying machine model. For example, consider the following bit of Java-like code: 
Object x = new Object();
Object y = new Object();
... some more code ...

So this program creates an object and names it x, and then creates a second object named y, and then continues executing some more code. Now, suppose that a programmer decides to flip the order of allocation of these two objects:
Object y = new Object();
Object x = new Object();
... some more code ...

Now, ask the question: does this refactoring change the behavior of the program? On the one hand, on the underlying machine, x and y will be allocated at different locations in the two runs of the program. So in this sense, the program behaves differently.
But in a Java-like language, you can only test references for equality, and not for order, so this is a difference that the "some more code" cannot observe.  As a result, most programmers will expect that reversing the order will make no difference to the final answer, and most compiler writers expect to be able to perform reorderings and optimizations on this basis. (On the other hand, in a C-like language, you can compare pointers for ordering, by casting them to integers first, and so this reordering does not necessarily preserve observable behavior.) 
One of the central questions of semantics is to answer the question of when two programs are observably equivalent. Since our notion of observation depends on the features of the programming language, we end up with a definition like "two programs are equivalent when no client program can compute different answers based on receiving those programs as inputs." The quantification over all client programs is what makes this question difficult -- it seems like you end up having to say something about all possible client programs to say something about two particular pieces of code.
The trick with denotational semantics is to give a mathematical interpretation that lets you avoid this universal quantification -- you say that the meaning of a piece of code is some mathematical value, and you compare them by checking to see if they're mathematically equal or not. This is local (ie, compositional), and does not involve quantification over all possible clients. (You do need to show that the denotational semantics implies contextual equivalence for it to be sound, of course. When it is complete -- when denotational equality is exactly the same as contextual equivalence, we say the semantics is "fully abstract".)
But means that you need to ensure that the denotational semantics validates those equivalences. So for this example, if you wanted to give a denotational semantics for this Java-like language, you need to ensure not just that calling new takes a heap and gives you back a new heap with the newly-created object, but that the meaning of the program is invariant same under all permutations of the input heap. This can involve quite complex mathematical structures (eg, in this case working in a category which ensures everything works modulo a suitable permutation group).

> **Comments:**
> **username_1**: "two programs are equivalent when no client program can compute different answers based on receiving those programs as inputs." I'm confused by this. If you have a program X and a *client* program Y, then I take it to mean that Y 'calls into' X. But then you seem to say that Y reads the text of X as *input*, in which case I'd hardly call Y a 'client' of X. Could you please clarify?  
> **username_6**: By "client of X", I just mean the same as "program context", which is just a "larger program which contains X as a subterm".  
> **username_1**: So you use 'X is a client of Y' interchangeably with 'X reads Y as input' because you think of X as a lambda applied to Y? It makes sense, but it's a little twisted when you talk about Java. :)  
> **username_7**: @RaduGRIGore: program context means something else. You are reading correctly the post, but if X reads the source code of Y as input (which is how I interpret the post), you can distinguish every two syntactically different programs; instead if Y is a lambda function on X, you can distinguish too few programs. Neel's comment about "program context" is the correct definition: a program context Y is a program with a hole in its AST, where you can place (meaningfully) two different program fragments X1 and X2.  
> **username_7**: @NeelKrishnaswami: could you maybe clarify what you mean in the post? You can just continue using your example and talk about a program where you can insert one or the other fragment.  

---

### Answer by username_3 on 2010-08-29. Score: 13

Matthias Felleisen presented a compelling solution to the side effects problem in semantics in his series on "Syntactic Theories of Control and State."

That line of work resulted in the CESK machine, a simple abstract machine framework capable of concisely modeling functional, object-oriented, imperative and even logic languages. The CESK framework handles not just side effects, but also "complex" control constructs like exceptions, continuations, laziness and even threads.

The CESK machine, and small-step operational semantics more broadly, have been the de facto standard in programming language theory for about two decades.

In short, a CESK machine is a small-step machine with four components to describe every machine state: the control string (a generalization of the program counter), the environment, a store (also called the heap) and the current continuation.

The environment maps variables to addresses; the store maps addresses to values.

This makes it straightforward to model mutable variables: just change the value at its address.

It also makes it easy to model pointers and dynamic allocation: just make store addresses first-class values.

In a similar fashion, first-class continuations result from making them addressable values.


---

### Answer by username_4 on 2010-08-28. Score: 12

There are of course ways of dealing with effects in (denotational) semantics. For example, we can use [Eugenio Moggi's](http://www.disi.unige.it/person/MoggiE/) idea that [computational effects are monads](http://www.disi.unige.it/person/MoggiE/ftp/ic91.pdf) (this idea has also been used in the design of Haskell). One of the problems with this is that monads are hard to combine. [Gordon Plotkin](http://homepages.inf.ed.ac.uk/gdp/) and [John Power](http://www.bath.ac.uk/comp-sci/people/contact/index.php?contact=Dr_John_Power) suggested [a refinement of Moggi's monads to Lawvere theories](http://homepages.inf.ed.ac.uk/gdp/publications/Overview.pdf), or algebraic theories as they are also called, which encompasses *algebraic* effects (most common effects are algebraic, such as state, I/O, non-determinism, but continuations are not). For a comprehensive treatment, see [Matija Pretnar's thesis](http://matija.pretnar.info/handlers-of-algebraic-effects.pdf).

I should also mention the [possible worlds semantics](http://portal.acm.org/citation.cfm?id=910747) for local state, developed by Frank Oles and [John Reynolds](http://www.cs.cmu.edu/~jcr/) (sorry, can't find a better link, this stuff is from 1982), which predates Moggi's monads. They used categories of presheaves to provide a semantics of an algol-like language which correctly modeled many aspects of local state (but not all of them, I think the model allowed snapback, but maybe my memory serves me wrong).

> **Comments:**
> **username_6**: Yes, the functor category semantics didn't validate all of the Meyer-Sieber equivalences. Peter O'Hearn and Robert Tennant developed a parametric version of the functor-category semantics in the mid-90s which (IIRC) got all of the Meyer-Sieber examples, but I don't know if it was fully abstract or not.  
> **username_5**: O'Hearn and Tennent model is not fully abstract.  That is discussed in the paper itself.  But the refinement by O'Hearn and Reynolds using linear lambda calculus is fully abstract up to second-order.  It breaks for third-order, examples being the equivalences studied by Ahmed, Dreyer, Birkedal et al.  

---

### Answer by username_2 on 2010-08-27. Score: 6

*How does the existence of side effects in a programming language impact the ability to map a program to a computational model?*

It doesn't necessarily make it difficult, but it does impose restrictions on the way the semantics of larger expressions can be constructed out of smaller ones. It can interact very badly with certain other programming constructs, for instance, if one wants to give a Scott-style denotational semantics for a language allowing assignment of higher-order functions to global references.

It is not simply side effects like state that cause the trouble. Simple imperative languages such as Dijkstra's guarded command language have these kinds of side effects, and have nice semantics. Trouble arises with extensions of the lambda-calculus with the kind of operational semantics expected of programming languages even in the absence of side effects: the earliest, Plotkin's PCF, was given denotational models relatively early, but the semantics were not fully abstract, meaning that the denotational semantics was overly general, not exactly corresponding to their operational semantics. PCF finally recieved a fully abstract denotational semantics in the late 1980s with game semantics, which is not at all like Scott's order-theoretic semantics. Concurrency still has not received a fully adequate denotational treatment.

Many question the importance of this kind of semantics. We can always provide some kind of operational semantics, even if that "semantics" is just the program source and the names of some machines that have compiled and run the program: for this reason Strachey condemned operational semantics. But Plotkin's structural operation semantics has shown how operational semantics can be separated from machine models, and Pitt's work has shown how such semantics can support similar reasoning about programs and programming languages to denotational semantics. Thus operational semantics is a viable alternative to denotational semantics, and have been applied with success to a substantial number of programming languages such as Standard ML.

*Are there approaches to managing state that can improve this process while still allowing side effects?*

To some extent, difficulties providing semantics correspond to the difficulty of providing powerful programming languages that behave in the way one would expect. Pragmatically motivated design decisions —such as avoiding the use of global state together with concurrency, typically through message-passing concurrency— make it easier to provide semantics.

> **Comments:**
> **username_4**: Scott's PCF does not have state and neither is it Scott's, is it?See http://en.wikipedia.org/wiki/Programming_language_for_Computable_Functions  
> **username_2**: @Andrej: Err, quite, given that Luke Ong supervised my D.Phil, I should not make that mistake.  I posted a teaser-summary of Milner's PCF and Scott's LCF that is more ... succint than WP's as an LtU story: http://lambda-the-ultimate.org/node/2196  It occurs to me that you might have access to the missing Scott (1969) manuscript...  
> **username_4**: That would be Plotkin's PCF, I think :-) I can try to get hold of the manuscript, but I don't actually have it.  
> **username_5**: But the point remains that PCF doesn't have state.  What "reason" are you saying that Strachey to condemn operational semantics?  It wasn't apparent to me.  The last paragraph contradicts what you said earlier, viz., guarded commands have nice semantics but PCF doesn't!  
> **username_2**: @Andrej, Uday: I've fixed my post, less than three years later.  

---

