  We introduce PyTorch Geometric High Order (PyGHO), a library for High Order
Graph Neural Networks (HOGNNs) that extends PyTorch Geometric (PyG). Unlike
ordinary Message Passing Neural Networks (MPNNs) that exchange messages between
nodes, HOGNNs, encompassing subgraph GNNs and k-WL GNNs, encode node tuples, a
method previously lacking a standardized framework and often requiring complex
coding. PyGHO's main objective is to provide an unified and user-friendly
interface for various HOGNNs. It accomplishes this through streamlined data
structures for node tuples, comprehensive data processing utilities, and a
flexible suite of operators for high-order GNN methodologies. In this work, we
present a detailed in-depth of PyGHO and compare HOGNNs implemented with PyGHO
with their official implementation on real-world tasks. PyGHO achieves up to
$50\%$ acceleration and reduces the code needed for implementation by an order
of magnitude. Our library is available at
\url{https://github.com/GraphPKU/PygHO}.
