The random walk that is defined as $Y_{t} = Y_{t-1} + e_t$, where $e_t$ is white noise. Denotes that the current position is the sum of the previous position + an unpredicted term.
You can prove that the mean function $\mu_t = 0 $, since
$$E(Y_{t}) = E(e_1+ e_2+ ... +e_t) = E(e_1) + E(e_2) +... +E(e_t) = 0 + 0 + \cdots + 0$$
But why is it that the variance increases linearly with time?
Does this have something to do with it's not “pure” random, since the new position is very correlated with the previous one?
EDIT:
Now I have a much better understanding by visualizing a big sample of random walks, and here we can easily observe that the overall variance does increase over time,

and the mean is as expected around zero.
Maybe this was trivial after all, since in the very early stages of the time series (compare time = 10 with time = 100) the random walkers have not had yet the time to explore as much.

In short because it keeps adding the variance of the next increments to the variability we have in getting to where we are now. 

$\text{Var}(Y_{t}) = \text{Var}(e_1+ e_2+ ... +e_t)$
$\qquad\quad\;\;= \text{Var}(e_1) + \text{Var}(e_2) +... +\text{Var}(e_t)$  (independence)
$\qquad\quad\;\;= \sigma^2 + \sigma^2 + ... + \sigma^2=t\sigma^2\,,$

and we can see that $t\sigma^2$ increases linearly with $t$.



The mean is zero at each time point; if you simulated the series many times and averaged across series for a given time, that would average to something near 0



$\quad^{\text{Figure: 500 simulated random walks with sample mean in white and }}$
$\quad^{ \pm \text{ one standard deviation in red. Standard deviation increases with } \sqrt{t}\,.}$
