  In many large multiple testing problems the hypotheses are divided into
families. Given the data, families with evidence for true discoveries are
selected, and hypotheses within them are tested. Neither controlling the
error-rate in each family separately nor controlling the error-rate over all
hypotheses together can assure that an error-rate is controlled in the selected
families. We formulate this concern about selective inference in its
generality, for a very wide class of error-rates and for any selection
criterion, and present an adjustment of the testing level inside the selected
families that retains the average error-rate over the selected families.
