  Internet applications increasingly employ TCP not as a stream abstraction,
but as a substrate for application-level transports, a use that converts TCP's
in-order semantics from a convenience blessing to a performance curse. As
Internet evolution makes TCP's use as a substrate likely to grow, we offer
Minion, an architecture for backward-compatible out-of-order delivery atop TCP
and TLS. Small OS API extensions allow applications to manage TCP's send buffer
and to receive TCP segments out-of-order. Atop these extensions, Minion builds
application-level protocols offering true unordered datagram delivery, within
streams preserving strict wire-compatibility with unsecured or TLS-secured TCP
connections. Minion's protocols can run on unmodified TCP stacks, but benefit
incrementally when either endpoint is upgraded, for a backward-compatible
deployment path. Experiments suggest that Minion can noticeably improve
performance of applications such as conferencing, virtual private networking,
and web browsing, while incurring minimal CPU or bandwidth costs.
