  We introduce Onflow, a reinforcement learning technique that enables online
optimization of portfolio allocation policies based on gradient flows. We
devise dynamic allocations of an investment portfolio to maximize its expected
log return while taking into account transaction fees. The portfolio allocation
is parameterized through a softmax function, and at each time step, the
gradient flow method leads to an ordinary differential equation whose solutions
correspond to the updated allocations. This algorithm belongs to the large
class of stochastic optimization procedures; we measure its efficiency by
comparing our results to the mathematical theoretical values in a log-normal
framework and to standard benchmarks from the 'old NYSE' dataset. For
log-normal assets, the strategy learned by Onflow, with transaction costs at
zero, mimics Markowitz's optimal portfolio and thus the best possible asset
allocation strategy. Numerical experiments from the 'old NYSE' dataset show
that Onflow leads to dynamic asset allocation strategies whose performances
are: a) comparable to benchmark strategies such as Cover's Universal Portfolio
or Helmbold et al. "multiplicative updates" approach when transaction costs are
zero, and b) better than previous procedures when transaction costs are high.
Onflow can even remain efficient in regimes where other dynamical allocation
techniques do not work anymore. Therefore, as far as tested, Onflow appears to
be a promising dynamic portfolio management strategy based on observed prices
only and without any assumption on the laws of distributions of the underlying
assets' returns. In particular it could avoid model risk when building a
trading strategy.
