  Sign languages are multi-channel languages that communicate information
through not just the hands (manual signals) but also facial expressions and
upper body movements (non-manual signals). However, since automatic sign
language translation is usually performed by generating a single sequence of
glosses, researchers eschew non-manual and co-occurring manual signals in favor
of a simplified list of manual glosses. This can lead to significant
information loss and ambiguity. In this paper, we introduce a new task named
multi-channel sign language translation (MCSLT) and present a novel metric,
SignBLEU, designed to capture multiple signal channels. We validated SignBLEU
on a system-level task using three sign language corpora with varied linguistic
structures and transcription methodologies and examined its correlation with
human judgment through two segment-level tasks. We found that SignBLEU
consistently correlates better with human judgment than competing metrics. To
facilitate further MCSLT research, we report benchmark scores for the three
sign language corpora and release the source code for SignBLEU at
https://github.com/eq4all-projects/SignBLEU.
