  [Context] Nowadays, many software systems include Artificial Intelligence
(AI) components and changes in the development environment have been known to
induce variability in an AI-based system. [Objective] However, how an
environment configuration impacts the variability of these systems is yet to be
explored. Understanding and quantifying the degree of variability due to such
configurations can help practitioners decide the best environment configuration
for the most stable AI products. [Method] To achieve this goal, we performed
experiments with eight different combinations of three key environment
variables (operating system, Python version, and CPU architecture) on 30
open-source AI-based systems using the Travis CI platform. We evaluate
variability using three metrics: the output of an AI component like an ML model
(performance), the time required to build and run a system (processing time),
and the cost associated with building and running a system (expense). [Results]
Our results indicate that variability exists in all three metrics; however, it
is observed more frequently with respect to processing time and expense than
performance. For example, between Linux and MacOS, variabilities are observed
in 23%, 96.67%, and 100% of the studied projects in performance, processing
time, and expense, respectively. [Conclusion] Our findings underscore the
importance of identifying the optimal combination of configuration settings to
mitigate performance drops and reduce retraining time and cost before deploying
an AI-based system.
