  We present a method for finding optimal controllers for unknown,
time-varying, dynamic systems which can be re-initialized from a given initial
condition repeatedly, in which the performance measure is available for
sampling with noise, but analytically unknown. Such systems are present
throughout industry where processes must be repeated many times, such as a
voltage source which is repeatedly turned on for a fraction of a second from
zero initial conditions and then turned off again, whose output must track a
specific trajectory, while the system's components are slowly drifting with
time due to temperature variations. For systems with convex cost functions we
prove that our algorithm will produce controllers that approach the minimal
cost, e.g., the cost minimizing LQR optimal controller that could have been
designed analytically had the system and objective function been known. We
demonstrate the algorithm's effectiveness with simulation studies of noisy and
time-varying systems.
