  The realism of digital avatars is crucial in enabling telepresence
applications with self-expression and customization. While physical simulations
can produce realistic motions for clothed humans, they require high-quality
garment assets with associated physical parameters for cloth simulations.
However, manually creating these assets and calibrating their parameters is
labor-intensive and requires specialized expertise. Current methods focus on
reconstructing geometry, but don't generate complete assets for physics-based
applications. To address this gap, we propose \papername,~a novel approach that
performs body and garment co-optimization using differentiable simulation. By
integrating physical simulation into the optimization loop and accounting for
the complex nonlinear behavior of cloth and its intricate interaction with the
body, our framework recovers body and garment geometry and extracts important
material parameters in a physically plausible way. Our experiments demonstrate
that our approach generates realistic clothing and body shape suitable for
downstream applications. We provide additional insights and results on our
webpage: https://people.csail.mit.edu/liyifei/publication/diffavatar/
