  We report the discovery of a new planetary system around the K giant $\eta$
Cet (HIP 5364, HD 6805) based on 118 high-precision optical radial velocities
taken at Lick Observatory since July 2000. Since October 2011 an additional
nine near-infrared Doppler measurements have been taken using the ESO CRIRES
spectrograph (VLT, UT1). The visible data set shows two clear periodicities.
Although we cannot completely rule out that the shorter period is due to
rotational modulation of stellar features, the infrared data show the same
variations as in the optical, which strongly supports that the variations are
caused by two planets. Assuming the mass of $\eta$ Cet to be 1.7 $M_\odot$, the
best edge-on coplanar dynamical fit to the data is consistent with two massive
planets ($m_b\sin i$ = 2.6 $\pm$ 0.2 $M_{\mathrm{Jup}}$, $m_c\sin i$ = 3.3
$\pm$ 0.2 $M_{\mathrm{Jup}}$), with periods of $P_b$ = 407 $\pm$ 3 days and
$P_c$ = 740 $\pm$ 5 days and eccentricities of $e_b$ = 0.12 $\pm$ 0.05 and
$e_c$ = 0.08 $\pm$ 0.03. We tested a wide variety of edge-on coplanar and
inclined planetary configurations for stability, which agree with the derived
radial velocities. We find that in certain coplanar orbital configurations with
moderate $e_b$ eccentricity, the planets can be effectively trapped in an
anti-aligned 2:1 mean motion resonance. A much larger non-resonant stable
region exists in low-eccentricity parameter space, although it appears to be
much farther from the best fit than the 2:1 resonant region. In all other
cases, the system is categorized as unstable or chaotic. Another conclusion
from the coplanar inclined dynamical test is that the planets can be at most a
factor of $\sim$ 1.4 more massive than their suggested minimum masses. This
stability constraint on the inclination excludes the possibility of two brown
dwarfs, and strongly favors a planetary system.
