  We develop a new modeling framework for Inter-Subject Analysis (ISA). The
goal of ISA is to explore the dependency structure between different subjects
with the intra-subject dependency as nuisance. It has important applications in
neuroscience to explore the functional connectivity between brain regions under
natural stimuli. Our framework is based on the Gaussian graphical models, under
which ISA can be converted to the problem of estimation and inference of the
inter-subject precision matrix. The main statistical challenge is that we do
not impose sparsity constraint on the whole precision matrix and we only assume
the inter-subject part is sparse. For estimation, we propose to estimate an
alternative parameter to get around the non-sparse issue and it can achieve
asymptotic consistency even if the intra-subject dependency is dense. For
inference, we propose an "untangle and chord" procedure to de-bias our
estimator. It is valid without the sparsity assumption on the inverse Hessian
of the log-likelihood function. This inferential method is general and can be
applied to many other statistical problems, thus it is of independent
theoretical interest. Numerical experiments on both simulated and brain imaging
data validate our methods and theory.
