Fast and accurate depth estimation using fringe projection patterns plays an important role in the fringe projection three-dimensional measurement approach. Herein, a recurrent residual convolutional neural network based on U-Net (R2U-Net) is introduced to solve the problem of depth estimation with an improved accuracy than single-frame fringe projection patterns, and the corresponding depth estimation method for fringe projection patterns is proposed, which is verified using simulated and experimental data. Results reveal that, for the simulated data, the error in the predicted results of the proposed approach is 1.71×10^-6, which is less than the error of 7.98×10^-6 corresponding to the U-Net method. For the experimental data, the error between the depth map predicted using the proposed method and the label decreases by 13% than that using the corresponding U-Net approach. Furthermore, compared with the existing U-Net depth map prediction method, the height distribution curve in the depth map obtained using the proposed approach exhibits a greater fitness with the label, which increases the accuracy of the three-dimensional measurement results from single-frame fringe patterns.