In the detection system, the long distance optical fiber vibration can be detected by phase-sensitive optical time domain reflect technique. However, the reflected signals include clutters and noises, which leads to a high false alarm rate. A two- level detection algorithm, which can maintain constant false alarm rate, is proposed, including cell average constant false alarm rate (CA- CFAR) and ordered statistics constant false alarm rate (OS- CFAR) which work in serial ways, and the detection speed of CA- CFAR and the performance of OS-CFAR are conserved, respectively. Monte Carlo simulation is applied to determine the threshold of twolevel detection. Finally, Monte Carlo simulation and the experiment on the scene are applied to analyze the performance of the algorithm, which proves the feasibility and effectiveness.