  Identifying the configuration of chess pieces from an image of a chessboard
is a problem in computer vision that has not yet been solved accurately.
However, it is important for helping amateur chess players improve their games
by facilitating automatic computer analysis without the overhead of manually
entering the pieces. Current approaches are limited by the lack of large
datasets and are not designed to adapt to unseen chess sets. This paper puts
forth a new dataset synthesised from a 3D model that is an order of magnitude
larger than existing ones. Trained on this dataset, a novel end-to-end chess
recognition system is presented that combines traditional computer vision
techniques with deep learning. It localises the chessboard using a RANSAC-based
algorithm that computes a projective transformation of the board onto a regular
grid. Using two convolutional neural networks, it then predicts an occupancy
mask for the squares in the warped image and finally classifies the pieces. The
described system achieves an error rate of 0.23% per square on the test set, 28
times better than the current state of the art. Further, a few-shot transfer
learning approach is developed that is able to adapt the inference system to a
previously unseen chess set using just two photos of the starting position,
obtaining a per-square accuracy of 99.83% on images of that new chess set. The
code, dataset, and trained models are made available online.
