Linguistic Initialization for Inductive Reasoning in Heterogeneous Knowledge Graphs

Abstract

Knowledge Graphs (KGs) provide explicit relational structure, while Large Language Models (LLMs) encode rich semantic knowledge. We propose a lightweight linguistic initialization strategy for heterogeneous link prediction that improves robustness under sparsity and imbalance. For each node, we construct a compact textual view combining intrinsic description and local neighborhood context, encode it with a pre-trained language model, and use the resulting embeddings to initialize a relation-aware GNN. This design preserves standard message passing while providing early semantically meaningful representations. Across multiple imbalance regimes and strict entity-to-entity cold-start settings, the proposed initialization consistently improves over random initialization and reduces degree-dependent degradation. Our results show that semantic grounding can be integrated into heterogeneous GNN pipelines with minimal architectural changes and strong empirical benefits.

Resources

Details