  We present an extensive powder and single-crystal neutron scattering
investigation of the crystallographic structure and magnetic order of the
frustrated metallic $f$-electron magnet UNi$_4$B. We carry out a full
refinement of the crystallographic structure and conclude that the
low-temperature lattice symmetry is orthorhombic (space group Pmm2; cell
parameters: $a = 6.963(4)~\r{A}$ , $b = 14.793(9)~\r{A}$ , $c =
17.126(8)~\r{A}$). We determine the magnetically ordered structure, concluding
that below $T_\mathrm{N} = 19.5~\mathrm{K}$ the material undergoes a transition
into a partially ordered antiferromagnetic state. The magnetic structure is
consistent with the existence of toroidal order in this material. We further
test the proposal of a second magnetic transition occurring at
$330~\mathrm{mK}$, concluding that the thermodynamic anomalies observed at
these temperatures do not reflect modifications of the magnetic structure. Our
study provides a consistent picture of the interrelationship of structural and
magnetic properties in the frustrated magnet UNi$_4$B previously unresolved.
