In order to improve classification effect of hyperspectral image, a classification algorithm with two levels is proposed based on spectral clustering and sparse representation. Pixels to be classified and its neighborhood pixels are divided into two parts by spectral clustering. The class of selected pixels is identified by the joint sparse representation model. This algorithm makes full use of hyperspectral image spectral and spatial information of hyperspectral images, and both of the two levels. Finally, the proposed algorithm is corrected with the spatial information, namely, neighboring pixels' class is associated and classification results is smoothed. Numerical experiments demonstrate that this algorithm has high classification accuracy, good stability and anti-noise performance.