The classification of hyperspectral images is one of the most important study fields. The spectral information is used in traditional classification of hyperspectral images, while the spatial correlativity information is ignored. To solve this problem, a novel model called spatial-spectral Chinese restaurant process (ssCRP) is proposed to cluster the hyperspectral images, which is an extension of Chinese restaurant process. Both the spatial and spectral information are considered in the modeling and inference of the method. The proposed model clusters the hyperspectral images better than tradional methods and satisfies the requirement of hyperspectral image clustering. Firstly, in order to consider both spatial and spectral information, a new similarity measurement is defined withthe exponential decay function based on the spatial distance and spectral angle among pixels. Then, each pixel is associated with a table based on the table construction by considering the similarity. Finally, each table is allocated with a dish which corresponds to a cluster. Thus, each pixel of the hyperspectral image is allocated with a clustering label. The true hyperspectral image collected by airborne visible infrared imaging spectrometer (AVIRIS) is used to evaluate the performance of our model. Experimental results indicate that the proposed model outperforms traditional K-means and ISODATA. Compared with those of the two methods, the result of the proposed model is more regular with lower salt-and-pepper effect with higher spatial consistency. The classification accuracy of the proposed model reaches to 63.57% and the Kappa coefficient is 0.632 3, much higher than those of K-means and ISODATA. Meanwhile, the edges of the result of our model are well preserved.