  Social determinants of health (SDOH) $-$ the myriad of circumstances in which
people live, grow, and age $-$ play an important role in health outcomes.
However, existing outcome prediction models often only use proxies of SDOH as
features. Recent open data initiatives present an opportunity to construct a
more comprehensive view of SDOH, but manually integrating the most relevant
data for individual patients becomes increasingly challenging as the volume and
diversity of public SDOH data grows. Large language models (LLMs) have shown
promise at automatically annotating structured data. Here, we conduct an
end-to-end case study evaluating the feasibility of using LLMs to integrate
SDOH data, and the utility of these SDOH features for clinical prediction. We
first manually label 700+ variables from two publicly-accessible SDOH data
sources to one of five semantic SDOH categories. Then, we benchmark performance
of 9 open-source LLMs on this classification task. Finally, we train ML models
to predict 30-day hospital readmission among 39k heart failure (HF) patients,
and we compare the prediction performance of the categorized SDOH variables
with standard clinical variables. Additionally, we investigate the impact of
few-shot LLM prompting on LLM annotation performance, and perform a metadata
ablation study on prompts to evaluate which information helps LLMs accurately
annotate these variables. We find that some open-source LLMs can effectively,
accurately annotate SDOH variables with zero-shot prompting without the need
for fine-tuning. Crucially, when combined with standard clinical features, the
LLM-annotated Neighborhood and Built Environment subset of the SDOH variables
shows the best performance predicting 30-day readmission of HF patients.
