  The goal of this work is background-robust continuous sign language
recognition. Most existing Continuous Sign Language Recognition (CSLR)
benchmarks have fixed backgrounds and are filmed in studios with a static
monochromatic background. However, signing is not limited only to studios in
the real world. In order to analyze the robustness of CSLR models under
background shifts, we first evaluate existing state-of-the-art CSLR models on
diverse backgrounds. To synthesize the sign videos with a variety of
backgrounds, we propose a pipeline to automatically generate a benchmark
dataset utilizing existing CSLR benchmarks. Our newly constructed benchmark
dataset consists of diverse scenes to simulate a real-world environment. We
observe even the most recent CSLR method cannot recognize glosses well on our
new dataset with changed backgrounds. In this regard, we also propose a simple
yet effective training scheme including (1) background randomization and (2)
feature disentanglement for CSLR models. The experimental results on our
dataset demonstrate that our method generalizes well to other unseen background
data with minimal additional training images.
