  Tverberg's theorem bounds the number of points $\mathbb{R}^d$ needed for the
existence of a partition into $r$ parts whose convex hulls intersect. If the
points are colored with $N$ colors, we seek partitions where each part has at
most one point of each color. In this manuscript, we bound the number of color
classes needed for the existence of partitions where the convex hulls of the
parts intersect even after any set of $t$ colors is removed. We prove
asymptotically optimal bounds for $t$ when $r \le d+1$, improve known bounds
when $r>d+1$, and give a geometric characterization for the configurations of
points for which $t=N-o(N)$.
