How do I calculate relative error when the true value is zero?
Say I have $x_{true} = 0$ and $x_{test}$. If I define relative error as:
$$\text{relative error} = \frac{x_{true}-x_{test}}{x_{true}}$$
Then the relative error is always undefined. If instead I use the definition:
$$\text{relative error} = \frac{x_{true}-x_{test}}{x_{test}}$$
Then the relative error is always 100%. Both methods seem useless. Is there another alternative?

There are many alternatives, depending on the purpose.



A common one is the "Relative Percent Difference," or RPD, used in laboratory quality control procedures.  Although you can find many seemingly different formulas, they all come down to comparing the difference of two values to their average magnitude:

$$d_1(x,y) = \frac{x - y}{(|x| + |y|)/2} = 2\frac{x - y}{|x| + |y|}.$$

This is a signed expression, positive when $x$ exceeds $y$ and negative when $y$ exceeds $x$.  Its value always lies between $-2$ and $2$.  By using absolute values in the denominator it handles negative numbers in a reasonable way.  Most of the references I can find, such as the New Jersey DEP Site Remediation Program Data Quality Assessment and Data Usability Evaluation Technical Guidance, use the absolute value of $d_1$ because they are interested only in the magnitude of the relative error.



A Wikipedia article on Relative Change and Difference observes that 

$$d_\infty(x,y) = \frac{|x - y|}{\max(|x|, |y|)}$$

is frequently used as a relative tolerance test in floating point numerical algorithms.  The same article also points out that formulas like $d_1$ and $d_\infty$ may be generalized to

$$d_f(x,y) = \frac{x - y}{f(x,y)}$$

where the function $f$ depends directly on the magnitudes of $x$ and $y$ (usually assuming $x$ and $y$ are positive).  As examples it offers their max, min, and arithmetic mean (with and without taking the absolute values of $x$ and $y$ themselves), but one could contemplate other sorts of averages such as the geometric mean $\sqrt{|x y|}$, the harmonic mean $2/(1/|x| + 1/|y|)$ and $L^p$ means $((|x|^p + |y|^p)/2)^{1/p}$.  ($d_1$ corresponds to $p=1$ and $d_\infty$ corresponds to the limit as $p\to \infty$.)  One might choose an $f$ based on the expected statistical behavior of $x$ and $y$.  For instance, with approximately lognormal distributions the geometric mean would be an attractive choice for $f$ because it is a meaningful average in that circumstance.



Most of these formulas run into difficulties when the denominator equals zero.  In many applications that either is not possible or it is harmless to set the difference to zero when $x=y=0$.

Note that all these definitions share a fundamental invariance property: whatever the relative difference function $d$ may be, it does not change when the arguments are uniformly rescaled by $\lambda \gt 0$:

$$d(x,y) = d(\lambda x, \lambda y).$$

It is this property that allows us to consider $d$ to be a relative difference.  Thus, in particular, a non-invariant function like

$$d(x,y) =?\ \frac{|x-y|}{1 + |y|}$$

simply does not qualify.  Whatever virtues it might have, it does not express a relative difference.



The story does not end here.  We might even find it fruitful to push the implications of invariance a little further. 

The set of all ordered pairs of real numbers $(x,y)\ne (0,0)$ where $(x,y)$ is considered to be the same as $(\lambda x, \lambda y)$ is the Real Projective Line $\mathbb{RP}^1$. In both a topological sense and an algebraic sense, $\mathbb{RP}^1$ is a circle.  Any $(x,y)\ne (0,0)$ determines a unique line through the origin $(0,0)$.  When $x\ne 0$ its slope is $y/x$; otherwise we may consider its slope to be "infinite" (and either negative or positive).  A neighborhood of this vertical line consists of lines with extremely large positive or extremely large negative slopes.  We may parameterize all such lines in terms of their angle $\theta = \arctan(y/x)$, with $-\pi/2 \lt \theta \le \pi/2$.  Associated with every such $\theta$ is a point on the circle,

$$(\xi, \eta) = (\cos(2\theta), \sin(2\theta)) = \left(\frac{x^2-y^2}{x^2+y^2}, \frac{2xy}{x^2+y^2}\right).$$

Any distance defined on the circle can therefore be used to define a relative difference.

As an example of where this can lead, consider the usual (Euclidean) distance on the circle, whereby the distance between two points is the size of the angle between them.  The relative difference is least when $x=y$, corresponding to $2\theta = \pi/2$ (or $2\theta = -3\pi/2$ when $x$ and $y$ have opposite signs).  From this point of view a natural relative difference for positive numbers $x$ and $y$ would be the distance to this angle:

$$d_S(x,y) = \left|2\arctan\left(\frac{y}{x}\right) - \pi/2\right|.$$

To first order, this is the relative distance $|x-y|/|y|$--but it works even when $y=0$.  Moreover, it doesn't blow up, but instead (as a signed distance) is limited between $-\pi/2$ and $\pi/2$, as this graph indicates:



This hints at how flexible the choices are when selecting a way to measure relative differences.
