An algorithm of image registration based on Hough transform and SURF features is proposed, to solve the problem that the accuracy is low in image registration when large angle rotation and translation occur, and the problem that traditional method is sensitive about local deformation and uneven illumination. Firstly, Hough transform is used to extract straight line features to estimate the rotation and translation parameters to achieve a rough image registration. Then, the image is divided into a few parts, and the SURF is used to do the precisely match sub-regionally. Finally, matching parameters are calculated and the registration is accomplished. Experiments show that the method can be used in the registration of IR and VI images, compared with traditional methods, and this method can achieve higher accuracy in the presence of a large angle rotation and translation and get a better result when localized deformation or uneven illumination is occurring.