  Amazon and other e-commerce sites must employ mechanisms to protect their
millions of customers from fraud, such as unauthorized use of credit cards. One
such mechanism is order fraud evaluation, where systems evaluate orders for
fraud risk, and either "pass" the order, or take an action to mitigate high
risk. Order fraud evaluation systems typically use binary classification models
that distinguish fraudulent and legitimate orders, to assess risk and take
action. We seek to devise a system that considers both financial losses of
fraud and long-term customer satisfaction, which may be impaired when incorrect
actions are applied to legitimate customers. We propose that taking actions to
optimize long-term impact can be formulated as a Reinforcement Learning (RL)
problem. Standard RL methods require online interaction with an environment to
learn, but this is not desirable in high-stakes applications like order fraud
evaluation. Offline RL algorithms learn from logged data collected from the
environment, without the need for online interaction, making them suitable for
our use case. We show that offline RL methods outperform traditional binary
classification solutions in SimStore, a simplified e-commerce simulation that
incorporates order fraud risk. We also propose a novel approach to training
offline RL policies that adds a new loss term during training, to better align
policy exploration with taking correct actions.
