  Path reasoning is a notable recommendation approach that models high-order
user-product relations, based on a Knowledge Graph (KG). This approach can
extract reasoning paths between recommended products and already experienced
products and, then, turn such paths into textual explanations for the user.
Unfortunately, evaluation protocols in this field appear heterogeneous and
limited, making it hard to contextualize the impact of the existing methods. In
this paper, we replicated three state-of-the-art relevant path reasoning
recommendation methods proposed in top-tier conferences. Under a common
evaluation protocol, based on two public data sets and in comparison with other
knowledge-aware methods, we then studied the extent to which they meet
recommendation utility and beyond objectives, explanation quality, and consumer
and provider fairness. Our study provides a picture of the progress in this
field, highlighting open issues and future directions. Source code:
\url{https://github.com/giacoballoccu/rep-path-reasoning-recsys}.
