  In this paper, we propose a joint transceiver design for single-carrier
frequency-domain equalization (SC-FDE) based multiple-input multiple-output
(MIMO) relay systems. To this end, we first derive the optimal minimum
mean-squared error linear and decision-feedback frequency-domain equalization
filters at the destination along with the corresponding error covariance
matrices at the output of the equalizer. Subsequently, we formulate the source
and relay precoding matrix design problem as the minimization of a family of
Schur-convex and Schur-concave functions of the mean-squared errors at the
output of the equalizer under separate power constraints for the source and the
relay. By exploiting properties of the error covariance matrix and results from
majorization theory, we derive the optimal structures of the source and relay
precoding matrices, which allows us to transform the matrix optimization
problem into a scalar power optimization problem. Adopting a high
signal-to-noise ratio approximation for the objective function, we obtain the
global optimal solution for the power allocation variables. Simulation results
illustrate the excellent performance of the proposed system and its superiority
compared to conventional orthogonal frequency-division multiplexing based MIMO
relay systems.
