Digital cameras usually build a colorimetric characterization model through target samples, which, however, will affect the mapping between the cameras’ response values and the colorimetric values. The existing selection methods for target samples ignore the generality of target samples, namely that the colorimetric characterization model built based on the optimized target samples should be applicable to any color samples. Therefore, we proposed an optimization method of colorimetric characterization targets for the digital cameras based on uniform color samples. First, uniform color samples were selected from the spectral reflectance of extensive real objects. Then, the colorimetric characterization model was combined to optimize the target samples among the uniform color samples. The results show that the optimization of target samples is subject to the data type and the colorimetric characterization model. Consequently, different optimized target samples are obtained, and the optimized target samples outperform the typical target samples in terms of colorimetric characterization. In addition, for the RAW data, an increase in the number of target samples does not necessarily improve the accuracy of a linear colorimetric characterization model, while for the sRGB data, more target samples will be beneficial for enhancing the performance of a polynomial colorimetric characterization model.