An algorithm of image salient object detection combined with deep learning is proposed based on an improved recurrent deep convolutional neural network with the cross-level feature fusion. The feature extraction of input images is performed through this improved recurrent deep convolutional neural network model. The cross-level joint framework is used for the feature fusion and thus the initial salient maps with high-level semantics features are generated. The saliency propagation is applied to the fusion of initial salient maps and low-level image features, and thus the structural information is obtained. The saliency propagation results are further optimized with the conditional random field and the final salient maps are realized. With the massive datasets, the proposed algorithm is tested and compared with other algorithms. The research results show that the proposed method is more robust than the existing algorithms in the image salient object detection of the complex scenes. Moreover, the integrity of the significant target detection is improved and the background is suppressed effectively.