  A nucleotide sequence 35 base pairs long can take
1,180,591,620,717,411,303,424 possible values. An example of systems biology
datasets, protein binding microarrays, contain activity data from about 40000
such sequences. The discrepancy between the number of possible configurations
and the available activities is enormous. Thus, albeit that systems biology
datasets are large in absolute terms, they oftentimes require methods developed
for rare events due to the combinatorial increase in the number of possible
configurations of biological systems. A plethora of techniques for handling
large datasets, such as Empirical Bayes, or rare events, such as importance
sampling, have been developed in the literature, but these cannot always be
simultaneously utilized. Here we introduce a principled approach to Empirical
Bayes based on importance sampling, information theory, and theoretical physics
in the general context of sequence phenotype model induction. We present the
analytical calculations that underlie our approach. We demonstrate the
computational efficiency of the approach on concrete examples, and demonstrate
its efficacy by applying the theory to publicly available protein binding
microarray transcription factor datasets and to data on synthetic
cAMP-regulated enhancer sequences. As further demonstrations, we find
transcription factor binding motifs, predict the activity of new sequences and
extract the locations of transcription factor binding sites. In summary, we
present a novel method that is efficient (requiring minimal computational time
and reasonable amounts of memory), has high predictive power that is comparable
with that of models with hundreds of parameters, and has a limited number of
optimized parameters, proportional to the sequence length.
