<p>Here are a few more ideas:</p>

<p>Point out that that only $2^{32}$ values can be represented in 32 bits (or $2^{64}$ values in 64 bits), while there are an infinite number of real numbers. Therefore, not all real numbers can be represented exactly. For example, if you take the smallest positive number that can be represented exactly and divide it by 2, you can't represent it exactly (by definition).</p>

<p>Explain that floating-point numbers are represented as the sum of powers of 2. Have them compute relatively easy values, such as $1\frac{3}{4} = 1 + \frac{1}{2} + \frac{1}{4}$. Ask them to represent $\frac{1}{3}$ as the sum of powers of 2. They will not be able to.</p>

CC BY-SA 3.0
2017-06-26T00:28:29.307Z
2017-06-26T14:05:08.933Z
2017-06-26T14:05:08.933Z
Ellen Spertus
<p>As a student myself, I had a course about how processing and encoding works where we had a detailed part about operations in different bases and how <code>floats</code> and <code>doubles</code> differ from one another by the number of bytes allocated, and how the computer's different ways of encoding floating numbers can be done by hand to get a grasp of it, but this wasn't until my third year, after having a first introduction to operations in different bases during my second year.</p>

<p>Only basic understanding of numbers in different bases and knowledge of the fact that <code>floats</code> are coded on a given number of bits is necessary to understand why it may be a problem, you could use the example of scientific calculations, where precision is important to make students get the stakes. But if you want students to really understand it fully, I think training operations in different bases and making them try by hand floating numbers coding and rounding may be necessary to get how frequent approximations are and how trivial calculations can give an answer different than what's expected, but more time consuming.</p>

<p>Also, if you want a playful and yet well explained way to teach students how those imprecisions work and how it may affect programs, here is a 10 min long video about floating numbers imprecision in mario 64 that's really well explained : <a href="https://www.youtube.com/watch?v=9hdFG2GcNuA" rel="nofollow noreferrer">https://www.youtube.com/watch?v=9hdFG2GcNuA</a></p>

CC BY-SA 3.0
2017-06-26T14:02:49.683Z
2017-06-27T14:29:25.827Z
2017-06-27T14:29:25.827Z
Romain Bernard
<p>There are plenty of good answers here, so I'll just add the extremely simple example I concocted to show why addition is not associative.</p>
<p>Suppose your have a decimal floating point system with one significant digit. Now what is <code>4+6+7</code>? If you evaluate left to right it's <code>2 * 10^1</code>, if you evaluate right to left it's <code>1 * 10^1</code>. (This is with rounding. I give my students a bonus exercise of coming up with an example that uses truncation.)</p>
<p>Worked out: <a href="https://pages.tacc.utexas.edu/%7Eeijkhout/istc/html/arithmetic.html#Associativity" rel="nofollow noreferrer">https://pages.tacc.utexas.edu/~eijkhout/istc/html/arithmetic.html#Associativity</a>  and read the whole page for anything else you want to know about floating point math.</p>

CC BY-SA 4.0
2021-12-27T17:48:43.140Z
2021-12-27T19:45:05.540Z
2021-12-27T19:45:05.540Z
Victor Eijkhout
<p>If your students can understand why:</p>

<p>$$\frac{1}{3} = 0.\dot{3}$$</p>

<p>then you can explain simply why floating point calculations are "broken".</p>

<p>When you divide one by three on a simple calculator, you can observe that it usually shows 0.333333333... until it runs out of digits (unless you have a more advanced calculator that can show recurring digits like <a href="https://i.ytimg.com/vi/BF74WoLp6NE/hqdefault.jpg" rel="noreferrer">this</a>).</p>

<p>You can explain that the phenomenon observed here is because of the same thing: one third in floating point is an <strong>infinitely repeating</strong> sequence:</p>

<p>$$ \frac{1}{3} = 0.0\overline{10}_2$$</p>

<p>If we could store that fully, then multiplying it by three would indeed give us one, as expected. But, like our naive calculator, we don't store all the bits in the sequence, so you end up with a slight inaccuracy. Pointing out that
$ 0.333333333 \times 3 = 0.999999999$ should be enough to illustrate the point that with an infinitely repeating decimal, you will not get the correct answer unless all the digits are taken account of (and you'll always be ever so slightly off the correct value, no matter which approximation you use).</p>

<p>It should then be clear to them that unless you use the exact value of $\frac{1}{3}$ in the binary representation, you will also get inaccuracies.</p>

<p>Something to note is that <strong>a finite decimal may be a repeating binary fraction</strong> (a significant example is $\frac{1}{10}$, which is $0.1_{10}$, but $0.0\overline{0011}_2$), but <a href="https://math.stackexchange.com/a/261989/378985">not vice versa</a>. Clearly, because of this, seemingly finite decimal numbers added together can lead to an infinite binary fraction, and hence an imprecise answer.</p>

<p>If your students are also familiar with <strong>scientific notation</strong>, then explaining the floating point system of a significant and mantissa should also be relatively painless. Since floating point can represent both <strong>very small</strong> and <strong>very large</strong> numbers, representing numbers as $\text{significand} \times 2^{\text{exponent}}$ is the most flexible option. You need not elaborate much further than that, though, I don't think. As <a href="https://cseducators.stackexchange.com/a/1100/2">Ben I.</a> mentions, a functional knowledge of floating point is needed, but the internals are probably not needed at this stage.</p>

<p>I don't see any reason why you <em>shouldn't</em> introduce the reason why floating point numbers are inaccurate to them quite early on — it's probably worse <em>not</em> to tell them, and leave them to be <strong>very</strong> confused when their program doesn't work because of the tiny inaccuracies ruining comparisons.</p>

<p>I think <a href="http://floating-point-gui.de/basic/" rel="noreferrer">floating-point-gui.de</a> does a great job at explaining, very concisely, without any implementation details (and you can read on about the details <strong>if you want</strong>). Using that as a resource may be helpful to find the right level to teach.</p>

<p>If you want to give your students something to play around with and get an understanding of the format of IEEE 754 floating point, you could try <a href="https://www.h-schmidt.net/FloatConverter/IEEE754.html" rel="noreferrer">this floating point number converter</a> which will show the representation of any number you enter.</p>

<p>It might also be wise to introduce a strategy to solve this problem—<a href="http://floating-point-gui.de/errors/comparison/" rel="noreferrer">epsilon values</a>. It should be relatively straightforward to explain that you just compare if a number is 'close' to what you expected with a floating point operation, rather than exactly equal; there are plenty of resources available that explain epsilon values further if necessary.</p>

CC BY-SA 3.0
2017-06-25T20:34:35.650Z
“If your students can understand why $1/3 = 0.\dot 3$” — that's actually a difficult one. I expect that most students don't really understand it.
Gilles 'SO- stop being evil'
CC BY-SA 3.0
2017-06-26T08:15:17.260Z
@Giles It's a very long time since I first met recurring decimals, but I don't recall too many fellow students not getting it.
TripeHound
CC BY-SA 3.0
2017-06-26T08:19:19.420Z
Once you've shown the problem with 1/3, it might be better to show 1/10 in binary. Although, as noted above, I could "get" recurring decimals, there's a danger of people thinking it's something peculiar about _one third_. Seeing the same problem happening with a "safe" number like _one tenth_ drove the point home for binary representations.
TripeHound
CC BY-SA 3.0
2017-06-26T12:52:53.247Z
+1, comment by @TripeHound about the dangers of "safe"/not-periodic numbers like 1/10 should be added to the answer. Also, [the problem with subtraction](https://stackoverflow.com/a/27539234/4209853)
xDaizu
CC BY-SA 3.0
2017-06-26T14:26:34.007Z
I would respectfully suggest that **this is a very misleading** example to illustrate the **computer science problem** of floating point precision. By using a repeating decimal as your example, you are (incorrectly) turning this into the math problem of representing fractions as decimals rather than a computer science problem that 1.0f + 1.0f =! 2.0f. This will leave students with a lot of bugs and head scratching trying to figure out why this doesn't work (i.e. not understanding how floating point numbers are represented).
Robert Cartaino
CC BY-SA 3.0
2017-06-26T15:53:05.093Z
@RobertCartaino You have an excellent point that using a number that is not finite in base-10 could lead to some false assumptions. I used the example from the question ($\frac{1}{3}$) when writing this, but I can see how it might be confusing. I think you cannot escape the fact that some numbers cannot be represented in binary, leading to these errors, which is **inherently a mathematical issue**, but I've added some more explicit information on how to solve this problem. Is there anything specifically that you feel ought to be mentioned here that I missed?
Aurora0001
CC BY-SA 3.0
2017-06-26T19:11:42.070Z
"a finite decimal may be a repeating binary fraction **and vice versa**" – Actually, a finite binary fraction will always convert to a finite decimal fraction (with at most the same number of digits).
Paŭlo Ebermann
CC BY-SA 3.0
2017-06-26T19:15:29.440Z
@PaŭloEbermann You're right. Thanks for pointing that out. I've amended that (and found a proof on Math.SE for anyone interested).
Aurora0001
CC BY-SA 3.0
2017-06-26T19:39:40.630Z
Something I'd suggest adding as an exercise with a regular calculator is that you can see the extra digits it hides and then uses in rounding to make everything seem to work out as expected.  Ex if after entering `1/7` your calculator displays `0.14285714", you can hit subtract and type in `0.142857141` hit enter and it'll show something like `2.8571E-9`.  With the exact amount depending on the calculators internal representation.
Dan Is Fiddling By Firelight
CC BY-SA 3.0
2017-06-26T19:41:08.450Z
@RobertCartaino I do not understand your objection here. 1.0 + 1.0 _does_ equal 2.0 in common floating-point systems. The problem is instead with 1.0 / 10.0 * 10.0, which rounds well in decimal but not in binary, and this answer explains why. Were you commenting on an earlier edit?
B. Szonye
CC BY-SA 3.0
2017-06-27T11:09:06.217Z
@Gilles - where in the world is this true? I'm in europe and that is literally a grade school material. The only way someone might not understand that is if they have a mental handicap.
Davor
CC BY-SA 3.0
2017-06-27T11:17:17.113Z
@Davor There's a huge gap between vaguely understanding that $1/3$ is not equal to $0.33$ and understanding infinitary decimal representation. Understanding that $0.\bar 9 = 1$ (as opposed to $0.\bar 9 \approx 1$) is a very difficult milestone. Having heard it in school is not the same thing as understanding it.
Gilles 'SO- stop being evil'
CC BY-SA 4.0
2021-12-27T17:42:55.000Z
Have you tried lately whether `3 * (1/3) -1` was zero? Current processors all have FMA units, which make this expression come out to zero! You need to do `3*3*(1/3)*(1/3)-1` to get a nonzero result.
Victor Eijkhout
CC BY-SA 3.0
2017-06-25T20:11:04.230Z
2017-06-26T19:14:21.367Z
2017-06-26T19:14:21.367Z
Aurora0001
<p>Even fairly early coders need to understand some limited form of why </p>

<pre><code>    double a = 1.000001;
    double b = 0.000001;
    System.out.println((a-b)==1.0);
</code></pre>

<p>ultimately prints <code>false</code>, and how to avoid this problem.</p>

<p>However, a more serious understanding of <strong>IEEE 754</strong> can wait.  I have my kids implement this standard (just the number representation, no arithmetic) as an elective after they have already learned about two's complement, big and little endian, and how a logical-gate 8-bit adder works.  Before they've had some time to live with and understand how binary numbers work, they won't have much of a chance of appreciating how floating point is dealt with on any serious level.</p>

CC BY-SA 3.0
2017-06-25T19:27:50.090Z
2017-06-25T19:27:50.090Z
Ben I.
<p>A few points:</p>

<ul>
<li>Do not call the discrepancies rounding errors</li>
<li>Do not confuse students with epsilon math</li>
<li>Do not obsess on teaching them exactly how IEEE floats are stored</li>
<li>Do not show them examples of the problem without explaining why things happened the way they did</li>
</ul>

<p>Teach them this:</p>

<p>Floats are confusing precisely because the way they are stored and the way they are represented is incompatible.  The values don't always translate 1 to 1. Some values that are easy to represent can't be stored faithfully with finite memory. So they are converted (not rounded) to the closest storable value available. Sometimes this goes completely unnoticed because when that stored value is converted (not rounded) back to a representation it ends up being the same representation it had before it went in. However, if math is done on this value then it's done on the stored value. The new stored value might get lucky and convert back to the correct representation. It might not. It's this "might" thing that makes this so hard. I can run tests on numbers thinking I got back what I put in so it must store exactly only to have my heart broken when I do math with it.</p>

<p>For example:</p>

<pre><code>1.03 - .42
</code></pre>

<p>gives</p>

<pre><code>0.6100000000000001
</code></pre>

<p>On their own 1.03 and .42 convert in and out of memory fine despite not being perfectly stored. Their difference is ALMOST right. It just doesn't happen to convert the way we hoped. Other math can put us at this same stored value that expects exactly this representation. There's no perfect way to round it. We simply don't know any more at this point than what was stored. </p>

<p>Yet:</p>

<pre><code>.01 - .02
</code></pre>

<p>gives a perfect</p>

<pre><code>-0.01
</code></pre>

<p>Why? Because we got lucky. The information that was lost in conversion just happened to not be enough to matter once it was converted back.</p>

<p>I hate problems that are hard to diagnose because I keep getting "lucky".</p>

<p>Now you can throw a lot of math and diagrams at this problem trying to explain how floats are stored but the thing to teach is how to know which numbers can't be stored perfectly.  Because the computer isn't going to tell you.</p>

<p>It's the students expectation of how floats will behave that you want to cultivate. They shouldn't expect them to behave like calculators. They shouldn't expect them to be useless. They should expect if you can't get to a number by addition and dividing by 2 that the numbers are going to be fuzzy. That's fine for scientific measurements that just want to be close. Not good for accounting that cares as much about pennies as dollars. A base 10 currency won't always store precisely. Though if you can find a currency that is always divided by 2 then floats will keep your accountant happy.</p>

<p><a href="https://i.sstatic.net/2j72n.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/2j72n.png" alt="ruler with inches"></a></p>

<p>Show them a ruler with inches on it. Floats can store inch fractions  perfectly. This should debunk the myth that floats have rounding errors. Entering these into a computer and doing math on them is a good exercise because it forces the student to do the conversion by hand. These conversions will all happen without discrepancies. You can even challenge the students to try to catch the computer doing inch fraction math wrong.</p>

<p>I used to teach this with drafting rulers. You can get one that has decimal inches. Put the decimal inch ruler next to the fraction inch ruler. Show them how some numbers match up but others don't.  The decimal inch ruler represents what you see and what you type. The fraction inch ruler represents what is stored.</p>

<p><a href="https://i.sstatic.net/R146f.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/R146f.png" alt="enter image description here"></a></p>

<p>You can even get students to reproduce the same kinds of errors by having them take measurements of values from the decimal ruler on the fraction ruler and trying to convert back. They'll quickly notice that some convert back easily while others don't. Well, as long as you don't let them simply remember how the story started.</p>

<p>You can find links to other resources in my earlier posts <a href="https://softwareengineering.stackexchange.com/a/344327/131624">here</a>. </p>

CC BY-SA 3.0
2017-06-27T07:24:30.050Z
I like how you stress the difference between rounding and conversion. However, your use of terminology is slightly incorrect compared to IEEE754. Conversion is the process of switching between decimal and float. In some cases, switching back and forth will not lose information. However, there is usually still rounding taking place in those conversions, possibly even if after converting back you end up with the same result (the two roundings simply cancelled out in that case). I would be hesitant to oversimplify things here, as that might do more harm than good in the long run.
ComicSansMS
CC BY-SA 3.0
2017-06-27T08:24:59.203Z
@ComicSansMS I'm trying to get away from the myth that the discrepancies in floats are just rounding errors.  I've seen to many students turn in code that attempts to correct the problem with their own rounding code. That's the oversimplification to avoid.
candied_orange
CC BY-SA 3.0
2017-06-27T13:45:11.420Z
The decimal inch ruler is perfect here.
Sean Houlihane
CC BY-SA 3.0
2017-06-26T18:57:39.607Z
2017-06-27T16:11:50.217Z
2017-06-27T16:11:50.217Z
candied_orange
<p>In my experience, the most confusing aspect here is not that there are rounding errors. Students should already have an understanding that there are physical limits that keep us from calculating with infinite precision all the time. What is confusing though is that these rounding errors occur in cases that might seem trivial to students.</p>

<p>What is important to understand here is that there are real numbers which do have a finite representation in decimal, but lead to an infinite representation when being converted to binary. <code>0.1</code> is probably the canonical example here. 
Students are used to thinking in base 10, so the most surprising rounding errors are those, where the operation still stays within the number of a couple of significant decimal digits, and therefore looks completely harmless when writing it down on paper, but still blows up because the numbers cannot be represented exactly in binary. Think: $0.1 + 0.2 - 0.3 \neq 0$.</p>

<p>Hammering this single point home should be enough to convince students that floats can lead to surprising rounding errors. The more challenging task is then to restore their faith in floats by explaining that the behavior of floats is still completely deterministic and one can actually give exact bounds for the expected rounding errors produced by a specific calculation.</p>

<p>For instance, many programming languages these days have support for <a href="https://stackoverflow.com/questions/4825824/hexadecimal-floating-constant-in-c">hex float literals</a>, which allow specifying float literals in base 16, eliminating potential rounding errors when converting the base 10 numbers from the source code to the base 2 numbers used by the machine. You still can get rounding errors on each calculation, but knowing what the numbers look like to the machine makes it much more predictable when to expect such rounding errors. Also, most of these calculation rounding errors are actually not due to the machine being unable to calculate exactly, but rather the fact that the result of the calculation can not be represented exactly in float. So the result is often really the closest to the correct result that you can possibly achieve with the chosen representation.</p>

CC BY-SA 3.0
2017-06-26T09:31:48.580Z
Firstly, I'd like to welcome you to Computer Science Educators! Secondly, this answer is superb, I do hope we will be hearing more from you.
ItamarG3
CC BY-SA 3.0
2017-06-26T13:49:09.177Z
I joined CSE just to up-vote this...
WLGfx
CC BY-SA 3.0
2017-06-27T09:47:04.117Z
If you need accuracy beyond the float standard, use integer math or a data type like [Decimal](https://stackoverflow.com/questions/1056323/difference-between-numeric-float-and-decimal-in-sql-server).
Cees Timmerman
CC BY-SA 4.0
2021-12-28T15:57:05.940Z
Nice answer, but have you actually tried whether `.1+.2 ?=? .3`? With clang11, putting the expression literally in a program the compiler probably optimizes it away, and yes, it is equal. I had to write `pointone + .2f -.3f` where `pointone` is read from `cin` to get a nonzero result. And even then, you only get nonzero with 2 out of the 4 rounding modes. And not the default one. So: Test Every Assertion You Make. This Crap Is Subtle! (My latest learning experience: `a * 1/a - 1` is zero on modern processors because this is an FMA instruction, which has single rounding.)
Victor Eijkhout
CC BY-SA 3.0
2017-06-26T08:09:46.570Z
2017-06-26T08:34:05.167Z
2017-06-26T08:34:05.167Z
ComicSansMS
<p>When I first started working with floating point arithmetic, issues like the one illustrated in the sample below both frustrated and confused me:</p>

<pre><code>scala&gt; (1f/3)*3d // admittedly a bit contrived using 1f
res0: Double = 1.0000000298023224
scala&gt; res0 == 1.0
res1: Boolean = false
</code></pre>

<p>I was used to mathematics, and a calculator which used fixed point arithmetic - an environment in which the answers were always "right" (to a given number of decimal places)! The way most programming languages handled decimal numbers felt <em>broken</em> and almost certainly useless for storing monetary values or for use in scientific code. I didn't appreciate the trade-off being made in terms of speed and space by taking the approach IEEE 754 did until much later on when I looked at the specification in detail and the analogy with standard scientific form was brought up.</p>

<p>At what stage should students be introduced to how floating point numbers are stored, and (more importantly) how can the motivations for this approach be properly conveyed?</p>

CC BY-SA 4.0
2019-04-18T11:06:15.347Z
What "inaccuracies?" You mean, results from erroneously deisgned chips? No? There are no "inaccuracies". So lesson number one is to not rely on thoughtless presumptions but read the specification for what you're using & lesson number 2 is there are two different abstractions involved in programming with floating point.
philipxy
CC BY-SA 4.0
2019-04-18T12:22:09.767Z
PS Show (better, have them graph/plot) an actual (normalized) float number line with its idiosyncratic spacing: `..._|______________|______________|______|_______|________|_______|_______|_______|_______|___|___|___|___|___|___|___|_|_|_|_|_|_|_||||||||________0________|||||||||_|_|_|_|_|_|_|___|___|___|___|___|___|___|______|_______|________|_______|_______|_______|_______|______________|______________|_...` [(1st figure in section 3_2 here)](http://www.toves.org/books/float/) What more need one say?
philipxy
CC BY-SA 3.0
2017-06-26T00:17:47.157Z
Here are some fun SMBC cartoons on the topic. http://www.smbc-comics.com/comic/2008-03-16 http://www.smbc-comics.com/?id=2999
Ellen Spertus
CC BY-SA 3.0
2017-06-26T15:39:17.217Z
Here is a YouTube video that gives an incredible explanation of floats: https://www.youtube.com/watch?v=9hdFG2GcNuA It's a speedrunner playing Super Mario 64 in an emulator, going into detail about how floating point numbers give rise to some unusual glitches in the game. The video is both educational and genuinely fascinating.
Kevin
CC BY-SA 3.0
2017-06-26T17:56:07.450Z
I realize that this doesn't change the fact that you need to explain float behavior to students but you should also consider the way BigDecimal works--it shouldn't have the same type of binary rounding problems because it bypasses the whole thing by using Integers for all decimal calculations.
Bill K
CC BY-SA 3.0
2017-06-26T19:09:06.237Z
@BillK the example in the question won't be better with BigDecimal. (Depending on how the division is done, you might get an exception there. But 1/3 is not representable exactly as a decimal number either.)
Paŭlo Ebermann
CC BY-SA 3.0
2017-06-26T19:36:19.480Z
How do you teach them the analogous **integer** problem (here, in C): `int i = 1/3; i = 3 * i; printf("%d\n", i);` -- surprise, the result is 0.
Pete Becker
CC BY-SA 3.0
2017-06-26T19:43:05.487Z
Actually I believe it will be better since it would be very obvious and predictable if the code fiddled with the precision (currently it casts it to float and double).  BigDecimal can't represent 1/3 exactly, but if you set it to, say 10 decimal places then every version of 1/3-1/3 should equal 0.  If you set one to 3 decimal places and the other to 4, you should always get a very predictable and sensible .0003.  The best part is you won't ever get rounding problems on numbers that shouldn't have rounding problems like .1
Bill K
CC BY-SA 3.0
2017-06-27T08:39:14.297Z
A useful model that keeps most of the floating point problems is a computer that works with two decimal places. So, you can exactly represent 75 and 0.12, but 114 is rounded to 110. Then you can go through practical examples, like finding the middle point between 92 and 94, that in the naive implementation is 95.
Davidmh
<number-system>
How should floating point inaccuracies be explained and justified?
# How should floating point inaccuracies be explained and justified?

(Asked by: username_0 on 2017-06-25)

When I first started working with floating point arithmetic, issues like the one illustrated in the sample below both frustrated and confused me:
scala> (1f/3)*3d // admittedly a bit contrived using 1f
res0: Double = 1.0000000298023224
scala> res0 == 1.0
res1: Boolean = false

I was used to mathematics, and a calculator which used fixed point arithmetic - an environment in which the answers were always "right" (to a given number of decimal places)! The way most programming languages handled decimal numbers felt broken and almost certainly useless for storing monetary values or for use in scientific code. I didn't appreciate the trade-off being made in terms of speed and space by taking the approach IEEE 754 did until much later on when I looked at the specification in detail and the analogy with standard scientific form was brought up.
At what stage should students be introduced to how floating point numbers are stored, and (more importantly) how can the motivations for this approach be properly conveyed?

**Tags:** number-system

> **Comments:**
> **username_2**: Here are some fun SMBC cartoons on the topic. http://www.smbc-comics.com/comic/2008-03-16 http://www.smbc-comics.com/?id=2999  
> **username_3**: Here is a YouTube video that gives an incredible explanation of floats: https://www.youtube.com/watch?v=9hdFG2GcNuA It's a speedrunner playing Super Mario 64 in an emulator, going into detail about how floating point numbers give rise to some unusual glitches in the game. The video is both educational and genuinely fascinating.  
> **username_4**: I realize that this doesn't change the fact that you need to explain float behavior to students but you should also consider the way BigDecimal works--it shouldn't have the same type of binary rounding problems because it bypasses the whole thing by using Integers for all decimal calculations.  
> **username_5**: @BillK the example in the question won't be better with BigDecimal. (Depending on how the division is done, you might get an exception there. But 1/3 is not representable exactly as a decimal number either.)  
> **username_6**: How do you teach them the analogous **integer** problem (here, in C): `int i = 1/3; i = 3 * i; printf("%d\n", i);` -- surprise, the result is 0.  
> **username_4**: Actually I believe it will be better since it would be very obvious and predictable if the code fiddled with the precision (currently it casts it to float and double).  BigDecimal can't represent 1/3 exactly, but if you set it to, say 10 decimal places then every version of 1/3-1/3 should equal 0.  If you set one to 3 decimal places and the other to 4, you should always get a very predictable and sensible .0003.  The best part is you won't ever get rounding problems on numbers that shouldn't have rounding problems like .1  
> **username_7**: A useful model that keeps most of the floating point problems is a computer that works with two decimal places. So, you can exactly represent 75 and 0.12, but 114 is rounded to 110. Then you can go through practical examples, like finding the middle point between 92 and 94, that in the naive implementation is 95.  
> **username_1**: What "inaccuracies?" You mean, results from erroneously deisgned chips? No? There are no "inaccuracies". So lesson number one is to not rely on thoughtless presumptions but read the specification for what you're using & lesson number 2 is there are two different abstractions involved in programming with floating point.  
> **username_1**: PS Show (better, have them graph/plot) an actual (normalized) float number line with its idiosyncratic spacing: `..._|______________|______________|______|_______|________|_______|_______|_______|_______|___|___|___|___|___|___|___|_|_|_|_|_|_|_||||||||________0________|||||||||_|_|_|_|_|_|_|___|___|___|___|___|___|___|______|_______|________|_______|_______|_______|_______|______________|______________|_...` [(1st figure in section 3_2 here)](http://www.toves.org/books/float/) What more need one say?  

---

## 7 answers

### Answer by username_10 on 2017-06-25. Score: 25 | Accepted answer

If your students can understand why:

$$\frac{1}{3} = 0.\dot{3}$$

then you can explain simply why floating point calculations are "broken".

When you divide one by three on a simple calculator, you can observe that it usually shows 0.333333333... until it runs out of digits (unless you have a more advanced calculator that can show recurring digits like [this](https://i.ytimg.com/vi/BF74WoLp6NE/hqdefault.jpg)).

You can explain that the phenomenon observed here is because of the same thing: one third in floating point is an **infinitely repeating** sequence:

$$ \frac{1}{3} = 0.0\overline{10}\_2$$

If we could store that fully, then multiplying it by three would indeed give us one, as expected. But, like our naive calculator, we don't store all the bits in the sequence, so you end up with a slight inaccuracy. Pointing out that
$ 0.333333333 \times 3 = 0.999999999$ should be enough to illustrate the point that with an infinitely repeating decimal, you will not get the correct answer unless all the digits are taken account of (and you'll always be ever so slightly off the correct value, no matter which approximation you use).

It should then be clear to them that unless you use the exact value of $\frac{1}{3}$ in the binary representation, you will also get inaccuracies.

Something to note is that **a finite decimal may be a repeating binary fraction** (a significant example is $\frac{1}{10}$, which is $0.1\_{10}$, but $0.0\overline{0011}\_2$), but [not vice versa](https://math.stackexchange.com/a/261989/378985). Clearly, because of this, seemingly finite decimal numbers added together can lead to an infinite binary fraction, and hence an imprecise answer.

If your students are also familiar with **scientific notation**, then explaining the floating point system of a significant and mantissa should also be relatively painless. Since floating point can represent both **very small** and **very large** numbers, representing numbers as $\text{significand} \times 2^{\text{exponent}}$ is the most flexible option. You need not elaborate much further than that, though, I don't think. As [username_18](https://cseducators.stackexchange.com/a/1100/2) mentions, a functional knowledge of floating point is needed, but the internals are probably not needed at this stage.

I don't see any reason why you *shouldn't* introduce the reason why floating point numbers are inaccurate to them quite early on — it's probably worse *not* to tell them, and leave them to be **very** confused when their program doesn't work because of the tiny inaccuracies ruining comparisons.

I think [floating-point-gui.de](http://floating-point-gui.de/basic/) does a great job at explaining, very concisely, without any implementation details (and you can read on about the details **if you want**). Using that as a resource may be helpful to find the right level to teach.

If you want to give your students something to play around with and get an understanding of the format of IEEE 754 floating point, you could try [this floating point number converter](https://www.h-schmidt.net/FloatConverter/IEEE754.html) which will show the representation of any number you enter.

It might also be wise to introduce a strategy to solve this problem—[epsilon values](http://floating-point-gui.de/errors/comparison/). It should be relatively straightforward to explain that you just compare if a number is 'close' to what you expected with a floating point operation, rather than exactly equal; there are plenty of resources available that explain epsilon values further if necessary.

> **Comments:**
> **username_11**: “If your students can understand why $1/3 = 0.\dot 3$” — that's actually a difficult one. I expect that most students don't really understand it.  
> **username_12**: @Giles It's a very long time since I first met recurring decimals, but I don't recall too many fellow students not getting it.  
> **username_12**: Once you've shown the problem with 1/3, it might be better to show 1/10 in binary. Although, as noted above, I could "get" recurring decimals, there's a danger of people thinking it's something peculiar about _one third_. Seeing the same problem happening with a "safe" number like _one tenth_ drove the point home for binary representations.  
> **username_13**: +1, comment by @username_12 about the dangers of "safe"/not-periodic numbers like 1/10 should be added to the answer. Also, [the problem with subtraction](https://stackoverflow.com/a/27539234/4209853)  
> **username_14**: I would respectfully suggest that **this is a very misleading** example to illustrate the **computer science problem** of floating point precision. By using a repeating decimal as your example, you are (incorrectly) turning this into the math problem of representing fractions as decimals rather than a computer science problem that 1.0f + 1.0f =! 2.0f. This will leave students with a lot of bugs and head scratching trying to figure out why this doesn't work (i.e. not understanding how floating point numbers are represented).  
> **username_10**: @RobertCartaino You have an excellent point that using a number that is not finite in base-10 could lead to some false assumptions. I used the example from the question ($\frac{1}{3}$) when writing this, but I can see how it might be confusing. I think you cannot escape the fact that some numbers cannot be represented in binary, leading to these errors, which is **inherently a mathematical issue**, but I've added some more explicit information on how to solve this problem. Is there anything specifically that you feel ought to be mentioned here that I missed?  
> **username_5**: "a finite decimal may be a repeating binary fraction **and vice versa**" – Actually, a finite binary fraction will always convert to a finite decimal fraction (with at most the same number of digits).  
> **username_10**: @PaŭloEbermann You're right. Thanks for pointing that out. I've amended that (and found a proof on Math.SE for anyone interested).  
> **username_15**: Something I'd suggest adding as an exercise with a regular calculator is that you can see the extra digits it hides and then uses in rounding to make everything seem to work out as expected.  Ex if after entering `1/7` your calculator displays `0.14285714", you can hit subtract and type in `0.142857141` hit enter and it'll show something like `2.8571E-9`.  With the exact amount depending on the calculators internal representation.  
> **username_16**: @RobertCartaino I do not understand your objection here. 1.0 + 1.0 _does_ equal 2.0 in common floating-point systems. The problem is instead with 1.0 / 10.0 * 10.0, which rounds well in decimal but not in binary, and this answer explains why. Were you commenting on an earlier edit?  
> **username_17**: @Gilles - where in the world is this true? I'm in europe and that is literally a grade school material. The only way someone might not understand that is if they have a mental handicap.  
> **username_11**: @username_17 There's a huge gap between vaguely understanding that $1/3$ is not equal to $0.33$ and understanding infinitary decimal representation. Understanding that $0.\bar 9 = 1$ (as opposed to $0.\bar 9 \approx 1$) is a very difficult milestone. Having heard it in school is not the same thing as understanding it.  
> **username_9**: Have you tried lately whether `3 * (1/3) -1` was zero? Current processors all have FMA units, which make this expression come out to zero! You need to do `3*3*(1/3)*(1/3)-1` to get a nonzero result.  

---

### Answer by username_20 on 2017-06-26. Score: 23

In my experience, the most confusing aspect here is not that there are rounding errors. Students should already have an understanding that there are physical limits that keep us from calculating with infinite precision all the time. What is confusing though is that these rounding errors occur in cases that might seem trivial to students.

What is important to understand here is that there are real numbers which do have a finite representation in decimal, but lead to an infinite representation when being converted to binary. `0.1` is probably the canonical example here.
Students are used to thinking in base 10, so the most surprising rounding errors are those, where the operation still stays within the number of a couple of significant decimal digits, and therefore looks completely harmless when writing it down on paper, but still blows up because the numbers cannot be represented exactly in binary. Think: $0.1 + 0.2 - 0.3 \neq 0$.

Hammering this single point home should be enough to convince students that floats can lead to surprising rounding errors. The more challenging task is then to restore their faith in floats by explaining that the behavior of floats is still completely deterministic and one can actually give exact bounds for the expected rounding errors produced by a specific calculation.

For instance, many programming languages these days have support for [hex float literals](https://stackoverflow.com/questions/4825824/hexadecimal-floating-constant-in-c), which allow specifying float literals in base 16, eliminating potential rounding errors when converting the base 10 numbers from the source code to the base 2 numbers used by the machine. You still can get rounding errors on each calculation, but knowing what the numbers look like to the machine makes it much more predictable when to expect such rounding errors. Also, most of these calculation rounding errors are actually not due to the machine being unable to calculate exactly, but rather the fact that the result of the calculation can not be represented exactly in float. So the result is often really the closest to the correct result that you can possibly achieve with the chosen representation.

> **Comments:**
> **username_22**: Firstly, I'd like to welcome you to Computer Science Educators! Secondly, this answer is superb, I do hope we will be hearing more from you.  
> **username_23**: I joined CSE just to up-vote this...  
> **username_24**: If you need accuracy beyond the float standard, use integer math or a data type like [Decimal](https://stackoverflow.com/questions/1056323/difference-between-numeric-float-and-decimal-in-sql-server).  
> **username_9**: Nice answer, but have you actually tried whether `.1+.2 ?=? .3`? With clang11, putting the expression literally in a program the compiler probably optimizes it away, and yes, it is equal. I had to write `pointone + .2f -.3f` where `pointone` is read from `cin` to get a nonzero result. And even then, you only get nonzero with 2 out of the 4 rounding modes. And not the default one. So: Test Every Assertion You Make. This Crap Is Subtle! (My latest learning experience: `a * 1/a - 1` is zero on modern processors because this is an FMA instruction, which has single rounding.)  

---

### Answer by username_18 on 2017-06-25. Score: 9

Even fairly early coders need to understand some limited form of why 
    double a = 1.000001;
    double b = 0.000001;
    System.out.println((a-b)==1.0);

ultimately prints false, and how to avoid this problem.
However, a more serious understanding of IEEE 754 can wait.  I have my kids implement this standard (just the number representation, no arithmetic) as an elective after they have already learned about two's complement, big and little endian, and how a logical-gate 8-bit adder works.  Before they've had some time to live with and understand how binary numbers work, they won't have much of a chance of appreciating how floating point is dealt with on any serious level.


---

### Answer by username_19 on 2017-06-26. Score: 9

A few points:

Do not call the discrepancies rounding errors
Do not confuse students with epsilon math
Do not obsess on teaching them exactly how IEEE floats are stored
Do not show them examples of the problem without explaining why things happened the way they did

Teach them this:
Floats are confusing precisely because the way they are stored and the way they are represented is incompatible.  The values don't always translate 1 to 1. Some values that are easy to represent can't be stored faithfully with finite memory. So they are converted (not rounded) to the closest storable value available. Sometimes this goes completely unnoticed because when that stored value is converted (not rounded) back to a representation it ends up being the same representation it had before it went in. However, if math is done on this value then it's done on the stored value. The new stored value might get lucky and convert back to the correct representation. It might not. It's this "might" thing that makes this so hard. I can run tests on numbers thinking I got back what I put in so it must store exactly only to have my heart broken when I do math with it.
For example:
1.03 - .42

gives
0.6100000000000001

On their own 1.03 and .42 convert in and out of memory fine despite not being perfectly stored. Their difference is ALMOST right. It just doesn't happen to convert the way we hoped. Other math can put us at this same stored value that expects exactly this representation. There's no perfect way to round it. We simply don't know any more at this point than what was stored. 
Yet:
.01 - .02

gives a perfect
-0.01

Why? Because we got lucky. The information that was lost in conversion just happened to not be enough to matter once it was converted back.
I hate problems that are hard to diagnose because I keep getting "lucky".
Now you can throw a lot of math and diagrams at this problem trying to explain how floats are stored but the thing to teach is how to know which numbers can't be stored perfectly.  Because the computer isn't going to tell you.
It's the students expectation of how floats will behave that you want to cultivate. They shouldn't expect them to behave like calculators. They shouldn't expect them to be useless. They should expect if you can't get to a number by addition and dividing by 2 that the numbers are going to be fuzzy. That's fine for scientific measurements that just want to be close. Not good for accounting that cares as much about pennies as dollars. A base 10 currency won't always store precisely. Though if you can find a currency that is always divided by 2 then floats will keep your accountant happy.

Show them a ruler with inches on it. Floats can store inch fractions  perfectly. This should debunk the myth that floats have rounding errors. Entering these into a computer and doing math on them is a good exercise because it forces the student to do the conversion by hand. These conversions will all happen without discrepancies. You can even challenge the students to try to catch the computer doing inch fraction math wrong.
I used to teach this with drafting rulers. You can get one that has decimal inches. Put the decimal inch ruler next to the fraction inch ruler. Show them how some numbers match up but others don't.  The decimal inch ruler represents what you see and what you type. The fraction inch ruler represents what is stored.

You can even get students to reproduce the same kinds of errors by having them take measurements of values from the decimal ruler on the fraction ruler and trying to convert back. They'll quickly notice that some convert back easily while others don't. Well, as long as you don't let them simply remember how the story started.
You can find links to other resources in my earlier posts here.

> **Comments:**
> **username_20**: I like how you stress the difference between rounding and conversion. However, your use of terminology is slightly incorrect compared to IEEE754. Conversion is the process of switching between decimal and float. In some cases, switching back and forth will not lose information. However, there is usually still rounding taking place in those conversions, possibly even if after converting back you end up with the same result (the two roundings simply cancelled out in that case). I would be hesitant to oversimplify things here, as that might do more harm than good in the long run.  
> **username_19**: @username_20 I'm trying to get away from the myth that the discrepancies in floats are just rounding errors.  I've seen to many students turn in code that attempts to correct the problem with their own rounding code. That's the oversimplification to avoid.  
> **username_21**: The decimal inch ruler is perfect here.  

---

### Answer by username_2 on 2017-06-26. Score: 4

Here are a few more ideas:

Point out that that only $2^{32}$ values can be represented in 32 bits (or $2^{64}$ values in 64 bits), while there are an infinite number of real numbers. Therefore, not all real numbers can be represented exactly. For example, if you take the smallest positive number that can be represented exactly and divide it by 2, you can't represent it exactly (by definition).

Explain that floating-point numbers are represented as the sum of powers of 2. Have them compute relatively easy values, such as $1\frac{3}{4} = 1 + \frac{1}{2} + \frac{1}{4}$. Ask them to represent $\frac{1}{3}$ as the sum of powers of 2. They will not be able to.


---

### Answer by username_8 on 2017-06-26. Score: 2

As a student myself, I had a course about how processing and encoding works where we had a detailed part about operations in different bases and how `floats` and `doubles` differ from one another by the number of bytes allocated, and how the computer's different ways of encoding floating numbers can be done by hand to get a grasp of it, but this wasn't until my third year, after having a first introduction to operations in different bases during my second year.

Only basic understanding of numbers in different bases and knowledge of the fact that `floats` are coded on a given number of bits is necessary to understand why it may be a problem, you could use the example of scientific calculations, where precision is important to make students get the stakes. But if you want students to really understand it fully, I think training operations in different bases and making them try by hand floating numbers coding and rounding may be necessary to get how frequent approximations are and how trivial calculations can give an answer different than what's expected, but more time consuming.

Also, if you want a playful and yet well explained way to teach students how those imprecisions work and how it may affect programs, here is a 10 min long video about floating numbers imprecision in mario 64 that's really well explained : <https://www.youtube.com/watch?v=9hdFG2GcNuA>


---

### Answer by username_9 on 2021-12-27. Score: 1

There are plenty of good answers here, so I'll just add the extremely simple example I concocted to show why addition is not associative.

Suppose your have a decimal floating point system with one significant digit. Now what is `4+6+7`? If you evaluate left to right it's `2 * 10^1`, if you evaluate right to left it's `1 * 10^1`. (This is with rounding. I give my students a bonus exercise of coming up with an example that uses truncation.)

Worked out: [https://pages.tacc.utexas.edu/~eijkhout/istc/html/arithmetic.html#Associativity](https://pages.tacc.utexas.edu/%7Eeijkhout/istc/html/arithmetic.html#Associativity) and read the whole page for anything else you want to know about floating point math.


---

