Remote sensing image has abundant granularity information. In order to utilize this information, a multiresolution region granularity analysis method is proposed in the present paper for image segmentation. The proposed method firstly uses the mean shift to obtain the initial over-segmented regions at each resolution of the image, and then extracts the granularity information based on the region size and the region context, the Markov random field is employed to provide the final segmentation result by modeling the spectrum information and the granularity information. The SPOT5 remote sensing images of Pingshuo and the aerial image of Taizhou were tested to evaluate the proposed method. Compared with other spectrum-based methods, our method shows a better performance and results improved the segmentation accuracy.