Structured light fringe projection is widely used in three-dimensional (3D) shape measurement technologies. However, even after decades of related research and development, restoring the 3D shape of objects using a single fringe pattern remains a challenge. In recent years, deep learning methods have been used increasingly in computer vision and image processing tasks. Thus, this paper proposes a structured light 3D measurement method based on the U-Net network. The proposed method obtains the depth information of the target object’s surface directly from a single deformed fringe pattern to realize high-speed 3D shape measurement. Simulated and experimental data prove the effectiveness of the proposed method. With the proposed method, 3D data can be calculated from a single deformed fringe pattern; thus, it can be applied to the 3D shape measurement of dynamic objects.