  Obtaining optimal data transfer performance is of utmost importance to
today's data-intensive distributed applications and wide-area data replication
services. Doing so necessitates effectively utilizing available network
bandwidth and resources, yet in practice transfers seldom reach the levels of
utilization they potentially could. Tuning protocol parameters such as
pipelining, parallelism, and concurrency can significantly increase utilization
and performance, however determining the best settings for these parameters is
a difficult problem, as network conditions can vary greatly between sites and
over time. Nevertheless, it is an important problem, since poor tuning can
cause either under- or over-utilization of network resources and thus degrade
transfer performance. In this paper, we present three algorithms for
application-level tuning of different protocol parameters for maximizing
transfer throughput in wide-area networks. Our algorithms dynamically tune the
number of parallel data streams per file (for large file optimization), the
level of control channel pipelining (for small file optimization), and the
number of concurrent file transfers to increase I/O throughput (a technique
useful for all types of files). The proposed heuristic algorithms improve the
transfer throughput up to 10x compared to the baseline and 7x compared to the
state of the art solutions.
