Diamond, a wide band gap semiconductor material, has been attracting interest in several fields from electrics and optics to biomedicine and quantum computing due to its outstanding properties. These properties of diamond are related to its unique lattice and optically active defect centers. In this paper, the dependence of nitrogen-vacancy (NV) center on measurement temperature is studied by using the low-temperature photoluminescence (PL) spectroscopy in a temperature range of 80–200 K. The results show that with the increase of the measurement temperature, the zero phonon lines of NV defects are red-shifted, its intensity decreases and its full width at half maximum increases. These results are attributed to the synergetic process of the lattice expansion and quadratic electron-phonon coupling. The NV^— and NV⁰ centers have similar values in the quenching activation energy and the thermal softening coefficient, resulting from their similar structures. The small differences may be associated with the electron-phonon coupling. The broadening mechanism of the NV centers is carefully distinguished by $T^3,\; T^5,\; T^7$ Voigt function fitting with the relation. These results show that the full width at half maximum of the Gaussian component of NV^— and NV⁰ centers are randomly distributed near 0.1 meV and 2.1 meV, respectively, while the full width at half maximum of the Lorentz component of NV^— and NV⁰ centers increase with measurement temperature increasing. The full width at half maximum of Lorentz of NV^— and NV⁰ centers conform to the $ T^3 $ relationship. It can be proved that under the action of the fluctuating field, the zero phonon lines of the NV defects exhibit an obvious homogeneous widening mechanism.