  Classical open systems with balanced gain and loss, i.e. parity-time
($\mathcal{PT}$) symmetric systems, have attracted tremendous attention over
the past decade. Their exotic properties arise from exceptional point (EP)
degeneracies of non-Hermitian Hamiltonians that govern their dynamics. In
recent years, increasingly sophisticated models of $\mathcal{PT}$-symmetric
systems with time-periodic (Floquet) driving, time-periodic gain and loss, and
time-delayed coupling have been investigated, and such systems have been
realized across numerous platforms comprising optics, acoustics, mechanical
oscillators, optomechanics, and electrical circuits. Here, we introduce a
$\mathcal{PT}$-symmetric (balanced gain and loss) system with memory, and
investigate its dynamics analytically and numerically. Our model consists of
two coupled $LC$ oscillators with positive and negative resistance,
respectively. We introduce memory by replacing either the resistor with a
memristor, or the coupling inductor with a meminductor, and investigate the
circuit energy dynamics as characterized by $\mathcal{PT}$-symmetric or
$\mathcal{PT}$-symmetry broken phases. Due to the resulting nonlinearity, we
find that energy dynamics depend on the sign and strength of initial voltages
and currents, as well as the distribution of initial circuit energy across its
different components. Surprisingly, at strong inputs, the system exhibits
self-organized Floquet dynamics, including $\mathcal{PT}$-symmetry broken phase
at vanishingly small dissipation strength. Our results indicate that
$\mathcal{PT}$-symmetric systems with memory show a rich landscape.
