In ac/ac converter applications, the matrix converter has become increasingly attractive in recent years. The matrix converter is a single stage converter. It does not require any dc-link energy storage component. So, the structure only requires small mounting place because the braking resistance or regeneration converter is unnecessary. In addition, it has a high-power-factor sinusoidal input current with a bidirectional power flow for the whole matrix converter control system. In this paper, the matrix converter is used to drive a PMSM servo motor. A sliding mode with a neural network controller design is proposed to improve the performance of the matrix converter-PMSM drive system. By using this scheme, the chattering of the speed is improved. In addition, the drive system has a better disturbance rejection capability and a fast speed response. All the control loops, including the switching strategy, current-loop, and control law, are implemented by TMS320LF2407A digital signal processor. The hardware circuit is very simple. Several experimental results are shown to validate the theoretical analysis.