A model of quantum dot refrigerator driven by photon, which consists of two two-level quantum dots, a photon reservoir and two leads, is proposed in this paper. Comparing with previous studies, we consider the transitions of electrons between different energy levels in a single quantum dot, which is more practical.Based on the theory of master equation and the assumption of weak coupling, we derive the expression of the cooling rate and the coefficient of performance of the refrigerator and obtain the condition of the tight coupling of the refrigerator operation. Next, we plot numerically the performance characteristic curves between the cooling rate and the coefficient of performance in the case of the tight coupling and in the general case. We find that the curves between the cooling rate and the coefficient of performance are opened loops for tight coupling, but they are closed loops in the general case. And we gain the conclusions that the refrigerator can be reversible under the condition of the tight coupling, while it can be irreversible in the general case. Then the optimally operating range of the refrigerator is determined. Finally, the effect of the temperature of the photon reservoir, transition coefficient, and temperature ratio on the performance of refrigerator under the conditions of the maximum cooling rate are studied, and also the coefficient of performance under the maximum cooling rate, the maximum coefficient of performanceand the cooling rate under the maximum coefficient of performanceare analyzed in detail.