  Two-level-system (TLS) defects in amorphous dielectrics are a major source of
noise and decoherence in solid-state qubits. Gate-dependent non-Markovian
errors caused by TLS-qubit coupling are detrimental to fault-tolerant quantum
computation and have not been rigorously treated in the existing literature. In
this work, we derive the non-Markovian dynamics between TLS and qubits during a
SWAP-like two-qubit gate and the associated average gate fidelity for
frequency-tunable Transmon qubits. This gate dependent error model facilitates
using qubits as sensors to simultaneously learn practical imperfections in both
the qubit's environment and control waveforms. We combine the-state-of-art
machine learning algorithm with Moir\'{e}-enhanced swap spectroscopy to achieve
robust learning using noisy experimental data. Deep neural networks are used to
represent the functional map from experimental data to TLS parameters and are
trained through an evolutionary algorithm. Our method achieves the highest
learning efficiency and robustness against experimental imperfections to-date,
representing an important step towards in-situ quantum control optimization
over environmental and control defects.
