  This paper presents novel methodologies for conducting practical
differentially private (DP) estimation and inference in high-dimensional linear
regression. We start by proposing a differentially private Bayesian Information
Criterion (BIC) for selecting the unknown sparsity parameter in DP-Lasso,
eliminating the need for prior knowledge of model sparsity, a requisite in the
existing literature. Then we propose a differentially private debiased LASSO
algorithm that enables privacy-preserving inference on regression parameters.
Our proposed method enables accurate and private inference on the regression
parameters by leveraging the inherent sparsity of high-dimensional linear
regression models. Additionally, we address the issue of multiple testing in
high-dimensional linear regression by introducing a differentially private
multiple testing procedure that controls the false discovery rate (FDR). This
allows for accurate and privacy-preserving identification of significant
predictors in the regression model. Through extensive simulations and real data
analysis, we demonstrate the efficacy of our proposed methods in conducting
inference for high-dimensional linear models while safeguarding privacy and
controlling the FDR.
