  Despite the acknowledged capability of template-free models in exploring
unseen reaction spaces compared to template-based models for retrosynthesis
prediction, their ability to venture beyond established boundaries remains
relatively uncharted. In this study, we empirically assess the extrapolation
capability of state-of-the-art template-free models by meticulously assembling
an extensive set of out-of-distribution (OOD) reactions. Our findings
demonstrate that while template-free models exhibit potential in predicting
precursors with novel synthesis rules, their top-10 exact-match accuracy in OOD
reactions is strikingly modest (< 1%). Furthermore, despite the capability of
generating novel reactions, our investigation highlights a recurring issue
where more than half of the novel reactions predicted by template-free models
are chemically implausible. Consequently, we advocate for the future
development of template-free models that integrate considerations of chemical
feasibility when navigating unexplored regions of reaction space.
