Transmission characteristic of a novel sub-wavelength grating color filter at visible region is investigated theoretically. The device consists of three parts: the flexible transparent substrate, the dielectric grating and the metal grating. By using rigorous couple-wave analysis (RCWA), the transmission characteristic is analyzed as a function of the duty cycle, the thickness of the dielectric grating, the thickness of the metal grating and the period. Based on the simulation results, a high-performance color filter is proposed by optimizing the structural parameters. The bandwidth of the filters are about 85 nm. The peak transmission efficiency of the filters are about 95%. The calculated results show that the peak transmission efficiency of the filters are increased more than 12% compared with the previous color filters incorporating a grating. The overlap of the tricolor output spectra decreases effectively, which will increase the color saturation of the color filter. The proposed color filter is suitable for filtering used in liquid crystal devices.