A Synthetic Aperture Radar (SAR) image retrieval algorithm is proposed based on semi-variances and Hu moments with rotation-invariant and flip-invariant. A set of semi-variances with selected lags and directions represents the image spatial continuity information. In order to achieve rotation-invariant and flip-invariant, the semi-variances were re-ordered according to the spatial major direction determined by weighted semi-variances proposed in this paper. The re-ordered semi-variances combined with 12 Hu moments represent SAR image features. Canberra distance was used to measure the similarity of normalized image features. Experiments were conducted on different scenes of rotated and flipped SAR images. Compared with algorithms based on multi-scale decomposition, the experiment results demonstrate that the proposed method is outstanding and more efficient than other algorithms.