  Model-free control strategies such as reinforcement learning have shown the
ability to learn control strategies without requiring an accurate model or
simulator of the world. While this is appealing due to the lack of modeling
requirements, such methods can be sample inefficient, making them impractical
in many real-world domains. On the other hand, model-based control techniques
leveraging accurate simulators can circumvent these challenges and use a large
amount of cheap simulation data to learn controllers that can effectively
transfer to the real world. The challenge with such model-based techniques is
the requirement for an extremely accurate simulation, requiring both the
specification of appropriate simulation assets and physical parameters. This
requires considerable human effort to design for every environment being
considered. In this work, we propose a learning system that can leverage a
small amount of real-world data to autonomously refine a simulation model and
then plan an accurate control strategy that can be deployed in the real world.
Our approach critically relies on utilizing an initial (possibly inaccurate)
simulator to design effective exploration policies that, when deployed in the
real world, collect high-quality data. We demonstrate the efficacy of this
paradigm in identifying articulation, mass, and other physical parameters in
several challenging robotic manipulation tasks, and illustrate that only a
small amount of real-world data can allow for effective sim-to-real transfer.
Project website at https://weirdlabuw.github.io/asid
