  We address the long-standing mystery of the nonmagnetic insulating state of
the intermediate valence compound SmB$_6$. Within a combination of the local
density approximation (LDA) and an exact diagonalization (ED) of an effective
discrete Anderson impurity model, the intermediate valence ground state with
the $f$-shell occupation $\langle n_{4f} \rangle=5.6$ is found for the Sm atom
in SmB$_6$. This ground state is a singlet, and the first excited triplet state
$\sim 3$ meV higher in the energy. SmB$_6$ is a narrow band insulator already
in LDA, with the direct band gap of $\sim 10$ meV. The electron correlations
increase the band gap which now becomes indirect. Thus, the many-body effects
are relevant to form the indirect band gap, crucial for the idea of
``topological Kondo insulator" in SmB$_6$. Also, an actinide analog PuB$_6$ is
considered, and the intermediate valence singlet ground state is found for the
Pu atom. We propose that [Sm,Pu]B$_6$ belong to a new class of the intermediate
valence materials with the multi-orbital ``Kondo-like" singlet ground-state.
Crucial role of complex spin-orbital $f^n$-$f^{n+1}$ multiplet structure
differently hybridized with ligand states in such Racah materials is discussed.
