  Payment channel networks (PCNs) are a layer-2 blockchain scalability
solution, with its main entity, the payment channel, enabling transactions
between pairs of nodes "off-chain," thus reducing the burden on the layer-1
network. Nodes with multiple channels can serve as relays for multihop payments
by providing their liquidity and withholding part of the payment amount as a
fee. Relay nodes might after a while end up with one or more unbalanced
channels, and thus need to trigger a rebalancing operation. In this paper, we
study how a relay node can maximize its profits from fees by using the
rebalancing method of submarine swaps. We introduce a stochastic model to
capture the dynamics of a relay node observing random transaction arrivals and
performing occasional rebalancing operations, and express the system evolution
as a Markov Decision Process. We formulate the problem of the maximization of
the node's fortune over time over all rebalancing policies, and approximate the
optimal solution by designing a Deep Reinforcement Learning (DRL)-based
rebalancing policy. We build a discrete event simulator of the system and use
it to demonstrate the DRL policy's superior performance under most conditions
by conducting a comparative study of different policies and parameterizations.
Our work is the first to introduce DRL for liquidity management in the complex
world of PCNs.
