  Compressing piecewise smooth images is important for many data types such as
depth maps in 3D videos or optic flow fields for motion compensation.
Specialised codecs that rely on explicitly stored segmentations excel in this
task since they preserve discontinuities between smooth regions. However,
current approaches rely on ad hoc segmentations that lack a clean
interpretation in terms of energy minimisation. As a remedy, we derive a
generic region merging algorithm from the Mumford-Shah cartoon model. It adapts
the segmentation to arbitrary reconstruction operators for the segment content.
In spite of its conceptual simplicity, our framework can outperform previous
segment-based compression methods as well as BPG by up to 3 dB.
