  In highly distributed environments such as cloud, edge and fog computing, the
application of machine learning for automating and optimizing processes is on
the rise. Machine learning jobs are frequently applied in streaming conditions,
where models are used to analyze data streams originating from e.g. video
streams or sensory data. Often the results for particular data samples need to
be provided in time before the arrival of next data. Thus, enough resources
must be provided to ensure the just-in-time processing for the specific data
stream. This paper focuses on proposing a runtime modeling strategy for
containerized machine learning jobs, which enables the optimization and
adaptive adjustment of resources per job and component. Our black-box approach
assembles multiple techniques into an efficient runtime profiling method, while
making no assumptions about underlying hardware, data streams, or applied
machine learning jobs. The results show that our method is able to capture the
general runtime behaviour of different machine learning jobs already after a
short profiling phase.
