  This paper targets the execution of data science (DS) pipelines supported by
data processing, transmission and sharing across several resources executing
greedy processes. Current data science pipelines environments provide various
infrastructure services with computing resources such as general-purpose
processors (GPP), Graphics Processing Units (GPUs), Field Programmable Gate
Arrays (FPGAs) and Tensor Processing Unit (TPU) coupled with platform and
software services to design, run and maintain DS pipelines. These one-fits-all
solutions impose the complete externalization of data pipeline tasks. However,
some tasks can be executed in the edge, and the backend can provide just in
time resources to ensure ad-hoc and elastic execution environments.
  This paper introduces an innovative composable "Just in Time Architecture"
for configuring DCs for Data Science Pipelines (JITA-4DS) and associated
resource management techniques. JITA-4DS is a cross-layer management system
that is aware of both the application characteristics and the underlying
infrastructures to break the barriers between applications,
middleware/operating system, and hardware layers. Vertical integration of these
layers is needed for building a customizable Virtual Data Center (VDC) to meet
the dynamically changing data science pipelines' requirements such as
performance, availability, and energy consumption. Accordingly, the paper shows
an experimental simulation devoted to run data science workloads and determine
the best strategies for scheduling the allocation of resources implemented by
JITA-4DS.
