A compact laser diode side-pumped multipass Nd:YVO4 slab laser amplifier is reported. The master oscillator is a passively Q-switched microchip laser, and the seed laser with average power of 0.5 W and pulse duration of 2.371 ns is obtained in TEM00 mode when the repetition frequency is 20 kHz. In order to achieve high efficiency and high gain amplification output, we design a compact structure to collimate the pump laser and the seed laser. In order to optimize the energy extraction and reduce the thermal aberrations, the pump laser is collimated by a cylindrical microlens in the vertical direction. In order to match the horizontal gain diameter, the spherical lenses (f=75 mm) are used to focus the laser into the slab in the first pass, and the cylindrical lenses (f=60 mm) are used to focus the laser into the slab in the second and the third passes. After three-pass amplification, average output power of 5.5 W, pulse duration of 2.52 ns and M2 of 1.5 are achieved when the repetition frequency is 20 kHz, and the corresponding energy extraction efficiency is greater than 14%.