Owing to its merits of high corrosion resistance, high temperature stability as well as good mechanical strength etc., silicon nitride membrane (SiN) has been widely used as the experimental carrier of transmission electron microscope (TEM), scanning electron microscopy (SEM), atomic force microscope (AFM), X-ray photoelectron spectroscopy (XPS), energy dispersive X-Ray spectroscopy (EDX) and other characterization. In particular, SiN can be used as a low disturbing background for SEM observation. However, the poor luminescent property of SiN thin film has restricted its wide application in fluorescent devices. In order to enhance the fluorescence efficiency of silicon nitride membrane, a series of ZnO films were prepared on a SiNx film substrate with radio frequency magnetron sputtering (RF magnetron sputtering) technology during the experiment. Samples were then non-situ and in-situ annealed in nitrogen atmosphere, respectively. Then, atomic force microscope (AFM), scanning electron microscopy (SEM) and Raman spectroscopy (Raman) were applied to study the microstructure and photoluminescence (PL) properties of the prepared films. This paper also systemically studies the luminescence of the prepared thin films. The results show that, luminescent intensity increases after sputtering, while annealing further promoted the grain growth, a substantial increase in crystallization behavior and a decrease in grain boundary. The microstructure and luminescence properties of ZnO/SiN thin films prepared by RF magnetron sputtering were significantly influenced by annealing method. Compared with the SiNx film, near the band edge of the intrinsic emission intensity (about 380 nm) of untempered ZnO/SiNx films and N2 atmosphere ex-situ annealed ZnO/SiNx films were increased by more than 77 times and 340 times. Compared with non-situ annealed films, in-situ annealed films contained more oxygen vacancy defects, thus showing a stronger visible light PL intensity. In-situ annealed films exhibited a higher photoluminescence capacity during the wavelength from 425 to 600 nm of visible light. These results can help to optimize the preparation parameters of silicon nitride based ZnO fluorescent films.