Traditional image segmentation methods mainly rely on the low-level features, such as image spectrum and texture, and are easily disturbed by occlusion and shadow. To address these problems, a CV (Chan-Vest) image segmentation model combining the convolutional restricted Boltzmann machine is proposed. The target shape a priori information is modeled and generated using the convolutional restricted Boltzmann machine. Then the energy function of the CV model is constrained by the added a priori shape term to guide image segmentation. Better segmentation results are obtained in remote sensing datasets Satellite-2000 and Vaihigen, whose training data are limited while target shapes and sizes are different.