  We present a novel computational model, "SAViR-T", for the family of visual
reasoning problems embodied in the Raven's Progressive Matrices (RPM). Our
model considers explicit spatial semantics of visual elements within each image
in the puzzle, encoded as spatio-visual tokens, and learns the intra-image as
well as the inter-image token dependencies, highly relevant for the visual
reasoning task. Token-wise relationship, modeled through a transformer-based
SAViR-T architecture, extract group (row or column) driven representations by
leveraging the group-rule coherence and use this as the inductive bias to
extract the underlying rule representations in the top two row (or column) per
token in the RPM. We use this relation representations to locate the correct
choice image that completes the last row or column for the RPM. Extensive
experiments across both synthetic RPM benchmarks, including RAVEN, I-RAVEN,
RAVEN-FAIR, and PGM, and the natural image-based "V-PROM" demonstrate that
SAViR-T sets a new state-of-the-art for visual reasoning, exceeding prior
models' performance by a considerable margin.
