  The non-Euclidean geometry of hyperbolic spaces has recently garnered
considerable attention in the realm of representation learning. Current
endeavors in hyperbolic representation largely presuppose that the underlying
hierarchies can be automatically inferred and preserved through the adaptive
optimization process. This assumption, however, is questionable and requires
further validation. In this work, we first introduce a position-tracking
mechanism to scrutinize existing prevalent \hlms, revealing that the learned
representations are sub-optimal and unsatisfactory. To address this, we propose
a simple yet effective method, hyperbolic informed embedding (HIE), by
incorporating cost-free hierarchical information deduced from the hyperbolic
distance of the node to origin (i.e., induced hyperbolic norm) to advance
existing \hlms. The proposed method HIE is both task-agnostic and
model-agnostic, enabling its seamless integration with a broad spectrum of
models and tasks. Extensive experiments across various models and different
tasks demonstrate the versatility and adaptability of the proposed method.
Remarkably, our method achieves a remarkable improvement of up to 21.4\%
compared to the competing baselines.
