  We provide tight upper and lower bounds on the expected minimum Kolmogorov
complexity of binary classifiers that are consistent with labeled samples. The
expected size is not more than complexity of the target concept plus the
conditional entropy of the labels given the sample.
