  Simulating vision-based tactile sensors enables learning models for
contact-rich tasks when collecting real world data at scale can be prohibitive.
However, modeling the optical response of the gel deformation as well as
incorporating the dynamics of the contact makes sim2real challenging. Prior
works have explored data augmentation, fine-tuning, or learning generative
models to reduce the sim2real gap. In this work, we present the first method to
leverage probabilistic diffusion models for capturing complex illumination
changes from gel deformations. Our tactile diffusion model is able to generate
realistic tactile images from simulated contact depth bridging the reality gap
for vision-based tactile sensing. On real braille reading task with a DIGIT
sensor, a classifier trained with our diffusion model achieves 75.74% accuracy
outperforming classifiers trained with simulation and other approaches. Project
page: https://github.com/carolinahiguera/Tactile-Diffusion
