An analytical expression is developed for the difference in buildup time between two adjacent longitudinal modes in a passively Q-switched laser resonator. The probability of single-frequency operation of both the small gain and the initial inversion population being several times greater than the threshold inversion can be predicted. A Cr4+YAG-passively Q-switched NdYAG laser set-up with 11 cm cavity length is used to carry out a series of experiments to verify the theoretical predictions. Experimental results support the probability of single mode operating based on adjacent longitudinal mode buildup-time difference, and the laser system can operate stably at single mode without any intra-cavity mode selecting elements in a long time in certain temperature range.