  Although a few methods have been developed recently for building confidence
intervals after model selection, how to construct confidence sets for joint
post-selection inference is still an open question. In this paper, we develop a
new method to construct confidence sets after lasso variable selection, with
strong numerical support for its accuracy and effectiveness. A key component of
our method is to sample from the conditional distribution of the response $y$
given the lasso active set, which, in general, is very challenging due to the
tiny probability of the conditioning event. We overcome this technical
difficulty by using estimator augmentation to simulate from this conditional
distribution via Markov chain Monte Carlo given any estimate $\tilde{\mu}$ of
the mean $\mu_0$ of $y$. We then incorporate a randomization step for the
estimate $\tilde{\mu}$ in our sampling procedure, which may be interpreted as
simulating from a posterior predictive distribution by averaging over the
uncertainty in $\mu_0$. Our Monte Carlo samples offer great flexibility in the
construction of confidence sets for multiple parameters. Extensive numerical
results show that our method is able to construct confidence sets with the
desired coverage rate and, moreover, that the diameter and volume of our
confidence sets are substantially smaller in comparison with a state-of-the-art
method.
