  Knowledge graph (KG) embedding methods which map entities and relations to
unique embeddings in the KG have shown promising results on many reasoning
tasks. However, the same embedding dimension for both dense entities and sparse
entities will cause either over parameterization (sparse entities) or under
fitting (dense entities). Normally, a large dimension is set to get better
performance. Meanwhile, the inference time grows log-linearly with the number
of entities for all entities are traversed and compared. Both the parameter and
inference become challenges when working with huge amounts of entities. Thus,
we propose PIE, a \textbf{p}arameter and \textbf{i}nference \textbf{e}fficient
solution. Inspired from tensor decomposition methods, we find that decompose
entity embedding matrix into low rank matrices can reduce more than half of the
parameters while maintaining comparable performance. To accelerate model
inference, we propose a self-supervised auxiliary task, which can be seen as
fine-grained entity typing. By randomly masking and recovering entities'
connected relations, the task learns the co-occurrence of entity and relations.
Utilizing the fine grained typing, we can filter unrelated entities during
inference and get targets with possibly sub-linear time requirement.
Experiments on link prediction benchmarks demonstrate the proposed key
capabilities. Moreover, we prove effectiveness of the proposed solution on the
Open Graph Benchmark large scale challenge dataset WikiKG90Mv2 and achieve the
state of the art performance.
