In the three-dimensional (3D) measurement system by the light-source-stepping method using LED, due to the large divergence angle of LED, the large fringe projection area can be achieved in a short distance, resulting in a short working distance of the system. In addition, the large luminous size of high-power LED leads to low fringe contrast, and it is difficult to project fringes with high brightness and high contrast. To solve the above problems, it is proposed to use laser diode (LD) as the light source to realize high brightness and high contrast phase-shift fringe projection in the light-source-stepping projection device. A portable and high-speed three-dimensional measurement system is designed by using the projection device and binocular camera. Firstly, an improved Fourier transform profilometry is used to obtain the phase and modulation of deformed fringe. Secondly, using the randomness of laser speckle, the rough matching of the binocular image is realized in the modulation map. Then, the fringe phase is unwrapped on the basis of rough matching, and the phase is used to realize fine matching. Finally, the three-dimensional shape of the object to be measured is obtained. The experimental verification is carried out by using the designed system. The measurement volume of the system is 360 mm×290 mm×100 mm, and the maximum 3D point captured is 1280×1024 points. The 3D shape measurement speed of 100 f/s was achieved. The measurement standard deviation of the plane is 0.19 mm. Five planes with a distance of 1.00 mm were measured, and the average error of the measurement distance is 0.05 mm.