Ten years ago, when I studied in university, I had no idea about definable numbers, but I came to this concept myself. My thoughts were as follows:

All numbers are divided into two classes: those which can be unambiguously defined by a limited set of their properties (definable) and those such that for any limited set of their properties there is at least one other number which also satisfies all these properties (undefinable).
It is evident that since the number of properties is countable, the set of definable numbers is countable. So the set of undefinable numbers forms a continuum.
It is impossible to give an example of an undefinable number and one researcher cannot communicate an undefinable number to the other. Whatever number of properties he communicates there is always another number which satisfies all these properties so the researchers cannot be confident whether they are speaking about the same number.
However there are probability based algorithms which give an undefinable number as a limit, for example, by throwing dice and writing consecutive numbers after the decimal point.

But the main question that bothered me was that the analysis course we received heavily relied on constructs such as "let $a$ be a number such that...", "for each $s$ in the interval..." etc. These seemed to heavily exploit the properties of definable numbers and as such one can expect the theorems of analysis to be correct only on the set of definable numbers. Even the definitions of arithmetic operations over reals assumed the numbers are definable. Unfortunately one cannot take an undefinable number to bring a counter-example just because there is no example of undefinable number. How can we know that all of those theorems of analysis are true for the whole continuum and not just for a countable subset?

The concept of definable real number, although seemingly
easy to reason with at first, is actually laden with subtle
metamathematical dangers to which both your question and
the Wikipedia article to which you link fall prey. In
particular, the Wikipedia article contains a number of
fundamental errors and false claims about this concept.    (Update, April 2018: The Wikipedia article, Definable real numbers, is now basically repaired and includes a link to this answer.)
The naive treatment of definability goes something like
this: In many cases we can uniquely specify a real number,
such as $e$ or $\pi$, by providing an exact description of
that number, by providing a property that is satisfied by
that number and only that number. More generally, we can
uniquely specify a real number $r$ or other set-theoretic
object by providing a description $\varphi$, in the formal
language of set theory, say, such that $r$ is the only
object satisfying $\varphi(r)$.
The naive account continues by saying that since there are
only countably many such descriptions $\varphi$, but
uncountably many reals, there must be reals that we cannot
describe or define.
But this line of reasoning is flawed in a number of ways
and ultimately incorrect. The basic problem is that the
naive definition of definable number does not actually
succeed as a definition. One can see the kind of problem
that arises by considering ordinals, instead of reals. That
is, let us suppose we have defined the concept of definable
ordinal; following the same line of argument, we would seem
to be led to the conclusion that there are only countably
many definable ordinals, and that therefore some ordinals
are not definable and thus there should be a least ordinal
$\alpha$ that is not definable. But if the concept of
definable ordinal were a valid set-theoretic concept, then
this would constitute a definition of $\alpha$, making a
contradiction. In short, the collection of definable
ordinals either must exhaust all the ordinals, or else not
itself be definable.
The point is that the concept of definability is a
second-order concept, that only makes sense from an
outside-the-universe perspective. Tarski's theorem on the
non-definability of
truth
shows that there is no first-order definition that allows
us a uniform treatment of saying that a particular
particular formula $\varphi$ is true at a point $r$ and
only at $r$. Thus, just knowing that there are only
countably many formulas does not actually provide us with
the function that maps a definition $\varphi$ to the object
that it defines. Lacking such an enumeration of the
definable objects, we cannot perform the diagonalization
necessary to produce the non-definable object.
This way of thinking can be made completely rigorous in the
following observations:

If ZFC is consistent, then there is a model of ZFC in
which every real number and indeed every set-theoretic
object is definable. This is true in the minimal
transitive model of set theory, by observing that the
collection of definable objects in that model is closed
under the definable Skolem functions of $L$, and hence by
Condensation collapses back to the same model, showing
that in fact every object there was definable.

More generally, if $M$ is
any model of ZFC+V=HOD, then the set $N$ of parameter-free
definable objects of $M$ is an elementary substructure of
$M$, since it is closed under the definable Skolem
functions provided by the axiom V=HOD, and thus every
object in $N$ is definable.


These models of set theory are pointwise definable,
meaning that every object in them is definable in them by a
formula. In particular, it is consistent with the axioms of
set theory that EVERY real number is definable, and indeed,
every set of reals, every topological space, every
set-theoretic object at all is definable in these models.

The pointwise definable models of set theory are
exactly the prime models of the models of ZFC+V=HOD, and
they all arise exactly in the manner I described above, as
the collection of definable elements in a model of V=HOD.

In recent work (soon to be submitted for publication),
Jonas Reitz, David Linetsky and I have proved the following
theorem:
Theorem. Every countable model of ZFC and indeed of
GBC has a forcing extension in which every set and class is
definable without parameters.
In these pointwise definable models, every object is
uniquely specified as the unique object satisfying a
certain property. Although this is true, the models also
believe that the reals are uncountable and so on, since
they satisfy ZFC and this theory proves that. The models
are simply not able to assemble the definability function
that maps each definition to the object it defines.
And therefore neither are you able to do this in general.
The claims made in both in your question and the Wikipedia
page on the existence of non-definable numbers and objects,
are simply unwarranted. For all you know, our set-theoretic
universe is pointwise definable, and every object is
uniquely specified by a property.

Update. Since this question was recently bumped to the main page by an edit to the main question, I am taking this opportunity to add a link to my very recent paper "Pointwise Definable Models of Set Theory", J. D. Hamkins, D. Linetsky, J. Reitz, which explains some of these definability issues more fully. The paper contains a generally accessible introduction, before the more technical material begins.
