  We develop a new statistical method for estimating functional connectivity
between neurophysiological signals represented by a multivariate time series.
We use partial coherence as the measure of functional connectivity. Partial
coherence identifies the frequency bands that drive the direct linear
association between any pair of channels. To estimate partial coherence, one
would first need an estimate of the spectral density matrix of the multivariate
time series. Parametric estimators of the spectral density matrix provide good
frequency resolution but could be sensitive when the parametric model is
misspecified. Smoothing-based nonparametric estimators are robust to model
misspecification and are consistent but may have poor frequency resolution. In
this work, we develop the generalized shrinkage estimator, which is a weighted
average of a parametric estimator and a nonparametric estimator. The optimal
weights are frequency-specific and derived under the quadratic risk criterion
so that the estimator, either the parametric estimator or the nonparametric
estimator, that performs better at a particular frequency receives heavier
weight. We validate the proposed estimator in a simulation study and apply it
on electroencephalogram recordings from a visual-motor experiment.
