  Abstract knowledge is deeply grounded in many computer-based applications. An
important research area of Artificial Intelligence (AI) deals with the
automatic derivation of knowledge from data. Machine learning offers the
according algorithms. One area of research focuses on the development of
biologically inspired learning algorithms. The respective machine learning
methods are based on neurological concepts so that they can systematically
derive knowledge from data and store it. One type of machine learning
algorithms that can be categorized as "deep learning" model is referred to as
Deep Neural Networks (DNNs). DNNs consist of multiple artificial neurons
arranged in layers that are trained by using the backpropagation algorithm.
These deep learning methods exhibit amazing capabilities for inferring and
storing complex knowledge from high-dimensional data. However, DNNs are
affected by a problem that prevents new knowledge from being added to an
existing base. The ability to continuously accumulate knowledge is an important
factor that contributed to evolution and is therefore a prerequisite for the
development of strong AIs. The so-called "catastrophic forgetting" (CF) effect
causes DNNs to immediately loose already derived knowledge after a few training
iterations on a new data distribution. Only an energetically expensive
retraining with the joint data distribution of past and new data enables the
abstraction of the entire new set of knowledge. In order to counteract the
effect, various techniques have been and are still being developed with the
goal to mitigate or even solve the CF problem. These published CF avoidance
studies usually imply the effectiveness of their approaches for various
continual learning tasks. This dissertation is set in the context of continual
machine learning with deep learning methods. The first part deals with the
development of an ...
