  The integration of artificial intelligence capabilities into modern software
systems is increasingly being simplified through the use of cloud-based machine
learning services and representational state transfer architecture design.
However, insufficient information regarding underlying model provenance and the
lack of control over model evolution serve as an impediment to the more
widespread adoption of these services in many operational environments which
have strict security requirements. Furthermore, tools such as TensorFlow
Serving allow models to be deployed as RESTful endpoints, but require
error-prone transformations for PyTorch models as these dynamic computational
graphs. This is in contrast to the static computational graphs of TensorFlow.
To enable rapid deployments of PyTorch models without intermediate
transformations we have developed FlexServe, a simple library to deploy
multi-model ensembles with flexible batching.
