Multi-sensor image registration is very difficult because of their different imaging principles. Referring to the registration of infrared and visible images, a registration algorithm is proposed based on the Harris corner with SIFT description. First, the scale spaces were built, and the scale-invariant Harris corners were selected as feature points. Then, by improving SIFT description method of feature points, ring operator was used to describe feature points. Finally, feature points were matched by the two-way nearest neighbor algorithm, and least squares method was employed to find optimal solutions to affine transform equations. Experimental results show that the registration algorithm is accurate, efficient and stable, and has a good registration result.