  We consider the $L^2$ critical inhomogeneous nonlinear Schr\"odinger (INLS)
equation in $\mathbb{R}^N$ $$ i \partial_t u +\Delta u +|x|^{-b}
|u|^{\frac{4-2b}{N}}u = 0, $$ where $N\geq 1$ and $0<b<2$. We prove that if
$u_0\in H^1(\mathbb{R}^N)$ satisfies $E[u_0]<0$, then the corresponding
solution blows-up in finite time. This is in sharp contrast to the classical
$L^2$ critical NLS equation where this type of result is only known in the
radial case for $N\geq 2$.
