  We compare different training strategies for the Deep Ritz Method for
elliptic equations with Dirichlet boundary conditions and highlight the
problems arising from the boundary values. We distinguish between an exact
resolution of the boundary values by introducing a distance function and the
approximation through a Robin Boundary Value problem. However, distance
functions are difficult to obtain for complex domains. Therefore, it is more
feasible to solve a Robin Boundary Value problem which approximates the
solution to the Dirichlet Boundary Value problem, yet the na\"ive approach to
this problem becomes unstable for large penalizations. A novel method to
compensate this problem is proposed using a small penalization strength to
pre-train the model before the main training on the target penalization
strength is conducted. We present numerical and theoretical evidence that the
proposed method is beneficial.
