  Tactile sensing is one of the modalities humans rely on heavily to perceive
the world. Working with vision, this modality refines local geometry structure,
measures deformation at the contact area, and indicates the hand-object contact
state.
  With the availability of open-source tactile sensors such as DIGIT, research
on visual-tactile learning is becoming more accessible and reproducible.
  Leveraging this tactile sensor, we propose a novel visual-tactile in-hand
object reconstruction framework \textbf{VTacO}, and extend it to
\textbf{VTacOH} for hand-object reconstruction. Since our method can support
both rigid and deformable object reconstruction, no existing benchmarks are
proper for the goal. We propose a simulation environment, VT-Sim, which
supports generating hand-object interaction for both rigid and deformable
objects. With VT-Sim, we generate a large-scale training dataset and evaluate
our method on it. Extensive experiments demonstrate that our proposed method
can outperform the previous baseline methods qualitatively and quantitatively.
Finally, we directly apply our model trained in simulation to various
real-world test cases, which display qualitative results.
  Codes, models, simulation environment, and datasets are available at
\url{https://sites.google.com/view/vtaco/}.
