  The thermal conductivity $\kappa$ of the Kondo insulator SmB$_6$ was measured
at low temperature, down to 70 mK, in magnetic fields up to 15 T, on single
crystals grown using both the floating-zone and the flux methods. The residual
linear term $\kappa_0/T$ at $T \to 0$ is found to be zero in all samples, for
all magnetic fields, in agreement with previous studies. There is therefore no
clear evidence of fermionic heat carriers. In contrast to some prior data, we
observe a large enhancement of $\kappa(T)$ with increasing field. The effect of
field is anisotropic, depending on the relative orientation of field and heat
current (parallel or perpendicular), and with respect to the cubic crystal
structure. We interpret our data in terms of heat transport predominantly by
phonons, which are scattered by magnetic impurities.
