  The new NASA Astrophysics Data System (ADS) is designed with a
serviceoriented architecture (SOA) that consists of multiple customized Apache
Solr search engine instances plus a collection of microservices, containerized
using Docker, and deployed in Amazon Web Services (AWS). For complex systems,
like the ADS, this loosely coupled architecture can lead to a more scalable,
reliable and resilient system if some fundamental questions are addressed.
After having experimented with different AWS environments and deployment
methods, we decided in December 2017 to go with Kubernetes as our container
orchestration. Defining the best strategy to properly setup Kubernetes has
shown to be challenging: automatic scaling services and load balancing traffic
can lead to errors whose origin is difficult to identify, monitoring and
logging the activity that happens across multiple layers for a single request
needs to be carefully addressed, and the best workflow for a Continuous
Integration and Delivery (CI/CD) system is not self-evident. We present here
how we tackle these challenges and our plans for the future.
