A novel photonic crystal (PC) structure with periodic rectangular holes is proposed to realize the narrow-band polarization notch filtering of visible lights. This PC structure is equivalent to a periodic structure with a dielectric-grating-dielectric layer in the PC structure model. The PC structure is first analyzed by the Rugate filtering theory and then the equivalent medium theory (EMT) and the transmission matrix method (TMM) are combined to simulate the transmittances of lights with s and p polarizations. The effects of the parameters, such as longitudinal period number m, filling ratio of grating layers f and thickness d of this PC structure, on the central wavelength, band width and transmittance in the cut-off zone for polarization notch filtering are also discussed. As for the central wavelengths of 417, 497, 582, 685 nm, a p-polarization notch filtering structure with a band width of 10 nm is designed, which is tested by the simulation with the finite difference time domain (FDTD) method. The results show that the PC structure with periodic rectangular holes can be used to realize the narrow-band polarization notch filtering of visible lights.