  Inferring gene regulatory networks (GRNs) from single-cell RNA sequencing
(scRNA-seq) data is a complex challenge that requires capturing the intricate
relationships between genes and their regulatory interactions. In this study,
we tackle this challenge by leveraging the single-cell BERT-based pre-trained
transformer model (scBERT), trained on extensive unlabeled scRNA-seq data, to
augment structured biological knowledge from existing GRNs. We introduce a
novel joint graph learning approach that combines the rich contextual
representations learned by pre-trained single-cell language models with the
structured knowledge encoded in GRNs using graph neural networks (GNNs). By
integrating these two modalities, our approach effectively reasons over boththe
gene expression level constraints provided by the scRNA-seq data and the
structured biological knowledge inherent in GRNs. We evaluate our method on
human cell benchmark datasets from the BEELINE study with cell type-specific
ground truth networks. The results demonstrate superior performance over
current state-of-the-art baselines, offering a deeper understanding of cellular
regulatory mechanisms.
