In recent years, fiber Bragg gratings (FBGs) have been widely used in ultrasonic detection for practical structural health monitoring in light of its unique advantages over the conventional sensors. Although FBGs have been successfully tested in ultrasonic inspection, the effect of the grating length on the sensitivity of the FBG ultrasonic sensing system is yet to be analyzed. Hence, using the simulation model, the main influencing factor on the sensitivity of the ultrasonic sensing system with different lengths gratings was first investigated. In the following experiment, the ultrasonic responses of the sensing system with six different lengths FBGs were obtained, respectively. The theoretical analysis and the experimental results would be useful for sensitivity improvement of the FBG-based ultrasonic and acoustic emission sensing system.