TV documentaries invariably show the Big Bang as an exploding ball of fire expanding outwards. Did the Big Bang really explode outwards from a point like this? If not, what did happen?
The simple answer is that no, the Big Bang did not happen at a point. Instead, it happened everywhere in the universe at the same time. Consequences of this include:

The universe doesn't have a centre: the Big Bang didn't happen at a point so there is no central point in the universe that it is expanding from.

The universe isn't expanding into anything: because the universe isn't expanding like a ball of fire, there is no space outside the universe that it is expanding into.


In the next section, I'll sketch out a rough description of how this can be, followed by a more detailed description for the more determined readers.
A simplified description of the Big Bang
Imagine measuring our current universe by drawing out a grid with a spacing of 1 light year. Although obviously, we can't do this, you can easily imagine putting the Earth at (0, 0), Alpha Centauri at (4.37, 0), and plotting out all the stars on this grid. The key thing is that this grid is infinite$^1$ i.e. there is no point where you can't extend the grid any further.
Now wind time back to 7 billion years after the big bang, i.e. about halfway back. Our grid now has a spacing of half a light year, but it's still infinite - there is still no edge to it. The average spacing between objects in the universe has reduced by half and the average density has gone up by a factor of $2^3$.
Now wind back to 0.0000000001 seconds after the big bang. There's no special significance to that number; it's just meant to be extremely small. Our grid now has a very small spacing, but it's still infinite. No matter how close we get to the Big Bang we still have an infinite grid filling all of space. You may have heard pop science programs describing the Big Bang as happening everywhere and this is what they mean. The universe didn't shrink down to a point at the Big Bang, it's just that the spacing between any two randomly selected spacetime points shrank down to zero.
So at the Big Bang, we have a very odd situation where the spacing between every point in the universe is zero, but the universe is still infinite. The total size of the universe is then $0 \times \infty$, which is undefined. You probably think this doesn't make sense, and actually, most physicists agree with you. The Big Bang is a singularity, and most of us don't think singularities occur in the real universe. We expect that some quantum gravity effect will become important as we approach the Big Bang. However, at the moment we have no working theory of quantum gravity to explain exactly what happens.
$^1$ we assume the universe is infinite - more on this in the next section
For determined readers only
To find out how the universe evolved in the past, and what will happen to it in the future, we have to solve Einstein's equations of general relativity for the whole universe. The solution we get is an object called the metric tensor that describes spacetime for the universe.
But Einstein's equations are partial differential equations, and as a result, have a whole family of solutions. To get the solution corresponding to our universe we need to specify some initial conditions. The question is then what initial conditions to use. Well, if we look at the universe around us we note two things:

if we average over large scales the universe looks the same in all directions, that is it is isotropic

if we average over large scales the universe is the same everywhere, that is it is homogeneous


You might reasonably point out that the universe doesn't look very homogeneous since it has galaxies with a high density randomly scattered around in space with a very low density. However, if we average on scales larger than the size of galaxy superclusters we do get a constant average density. Also, if we look back to the time the cosmic microwave background was emitted (380,000 years after the Big Bang and well before galaxies started to form) we find that the universe is homogeneous to about $1$ part in $10^5$, which is pretty homogeneous.
So as the initial conditions let's specify that the universe is homogeneous and isotropic, and with these assumptions, Einstein's equation has a (relatively!) simple solution. Indeed this solution was found soon after Einstein formulated general relativity and has been independently discovered by several different people. As a result the solution glories in the name Friedmann–Lemaître–Robertson–Walker metric, though you'll usually see this shortened to FLRW metric or sometimes FRW metric (why Lemaître misses out I'm not sure).
Recall the grid I described to measure out the universe in the first section of this answer, and how I described the grid shrinking as we went back in time towards the Big Bang? Well the FLRW metric makes this quantitative. If $(x, y, z)$ is some point on our grid then the current distance to that point is just given by Pythagoras' theorem:
$$ d^2 = x^2 + y^2 + z^2 $$
What the FLRW metric tells us is that the distance changes with time according to the equation:
$$ d^2(t) = a^2(t)(x^2 + y^2 + z^2) $$
where $a(t)$ is a function called the [scale factor]. We get the function for the scale factor when we solve Einstein's equations. Sadly it doesn't have a simple analytical form, but it's been calculated in answers to the previous questions What was the density of the universe when it was only the size of our solar system? and How does the Hubble parameter change with the age of the universe?. The result is:

The value of the scale factor is conventionally taken to be unity at the current time, so if we go back in time and the universe shrinks we have $a(t) < 1$, and conversely in the future as the universe expands we have $a(t) > 1$. The Big bang happens because if we go back to time to $t = 0$ the scale factor $a(0)$ is zero. This gives us the remarkable result that the distance to any point in the universe $(x, y, z)$ is:
$$ d^2(t) = 0(x^2 + y^2 + z^2) = 0 $$
so the distance between every point in the universe is zero. The density of matter (the density of radiation behaves differently but let's gloss over that) is given by:
$$ \rho(t) = \frac{\rho_0}{a^3(t)} $$
where $\rho_0$ is the density at the current time, so the density at time zero is infinitely large. At the time $t = 0$ the FLRW metric becomes singular.
No one I know thinks the universe did become singular at the Big Bang. This isn't a modern opinion: the first person I know to have objected publically was Fred Hoyle, and he suggested Steady State Theory to avoid the singularity. These days it's commonly believed that some quantum gravity effect will prevent the geometry from becoming singular, though since we have no working theory of quantum gravity no one knows how this might work.
So to conclude: the Big Bang is the zero time limit of the FLRW metric, and it's a time when the spacing between every point in the universe becomes zero and the density goes to infinity. It should be clear that we can't associate the Big Bang with a single spatial point because the distance between all points was zero so the Big Bang happened at all points in space. This is why it's commonly said that the Big Bang happened everywhere.
In the discussion above I've several times casually referred to the universe as infinite, but what I really mean is that it can't have an edge. Remember that our going-in assumption is that the universe is homogeneous i.e. it's the same everywhere. If this is true the universe can't have an edge because points at the edge would be different from points away from the edge. A homogenous universe must either be infinite, or it must be closed i.e. have the spatial topology of a 3-sphere. The recent Planck results show the curvature is zero to within experimental error, so if the universe is closed the scale must be far larger than the observable universe.
