  We address the problem of inferring the anatomic skeleton of a person, in an
arbitrary pose, from the 3D surface of the body; i.e. we predict the inside
(bones) from the outside (skin). This has many applications in medicine and
biomechanics. Existing state-of-the-art biomechanical skeletons are detailed
but do not easily generalize to new subjects. Additionally, computer vision and
graphics methods that predict skeletons are typically heuristic, not learned
from data, do not leverage the full 3D body surface, and are not validated
against ground truth. To our knowledge, our system, called OSSO (Obtaining
Skeletal Shape from Outside), is the first to learn the mapping from the 3D
body surface to the internal skeleton from real data. We do so using 1000 male
and 1000 female dual-energy X-ray absorptiometry (DXA) scans. To these, we fit
a parametric 3D body shape model (STAR) to capture the body surface and a novel
part-based 3D skeleton model to capture the bones. This provides inside/outside
training pairs. We model the statistical variation of full skeletons using PCA
in a pose-normalized space. We then train a regressor from body shape
parameters to skeleton shape parameters and refine the skeleton to satisfy
constraints on physical plausibility. Given an arbitrary 3D body shape and
pose, OSSO predicts a realistic skeleton inside. In contrast to previous work,
we evaluate the accuracy of the skeleton shape quantitatively on held-out DXA
scans, outperforming the state-of-the-art. We also show 3D skeleton prediction
from varied and challenging 3D bodies. The code to infer a skeleton from a body
shape is available for research at https://osso.is.tue.mpg.de/, and the dataset
of paired outer surface (skin) and skeleton (bone) meshes is available as a
Biobank Returned Dataset. This research has been conducted using the UK Biobank
Resource.
