Commission internationale de I′Eclairage (CIE) color rendering index has many deficiencies when applied to evaluating color quality of new-type light sources such as light emitting diode (LED). The universal evaluation method of light source color quality is discussed, which can play a vital role in contributing to the application of new-type light sources. 16 typical indexes are adopted to evaluate different types of illuminants and light sources in the experiment. The results indicate that most indexes are correlated with each other, and color fidelity based and gamut based methods are proposed to totally evaluate the color quality of light sources. An attempt is made and further verified to construct a comprehensive index of color quality for evaluating light sources.