  Chess recognition is the task of extracting the chess piece configuration
from a chessboard image. Current approaches use a pipeline of separate,
independent, modules such as chessboard detection, square localization, and
piece classification. Instead, we follow the deep learning philosophy and
explore an end-to-end approach to directly predict the configuration from the
image, thus avoiding the error accumulation of the sequential approaches and
eliminating the need for intermediate annotations. Furthermore, we introduce a
new dataset, Chess Recognition Dataset (ChessReD), that consists of 10,800 real
photographs and their corresponding annotations. In contrast to existing
datasets that are synthetically rendered and have only limited angles, ChessReD
has photographs captured from various angles using smartphone cameras; a sensor
choice made to ensure real-world applicability. Our approach in chess
recognition on the introduced challenging benchmark dataset outperforms related
approaches, successfully recognizing the chess pieces' configuration in 15.26%
of ChessReD's test images. This accuracy may seem low, but it is ~7x better
than the current state-of-the-art and reflects the difficulty of the problem.
The code and data are available through:
https://github.com/ThanosM97/end-to-end-chess-recognition.
