Based on the principle of inter mode interference, we apply the single mode-multimode-single mode (SMS) optical fiber structure to the optical fiber security system. In the system, multimode optical fiber is used as sensing fiber, and wavelet transform is used to realize the detection of external vibration signals. The theory of intermodal interference of multimode optical fiber is analyzed. The sensing optical fiber is arranged in the fence and laid on the ground, respectively. Several vibration invasive events are analyzed in experiment. The experimental results show that the sensing system of this structure can detect the vibration invasive signal quickly and effectively, and it has good feasibility in the field of security in optical fiber. The structure sensing system maintains the advantages of simple structure, low-cost, which has very important significance in optical fiber security systems.