An affine invariant derived from the second derivative matrix is studied and the affine invariant considering not only the differential characteristic of image but also the integral characteristic of image is constructed. First, the second derivative matrix of a given image and its eigenvectors and eigenvalues are computed. Second, the compactification of affine transformation is implemented and a set of ellipses with the same center controlled by the eigenvectors and eigenvalues of the second derivative matrix are constructed. Third, the affine invariant property of the ellipses with the same center is proved, and then a set of affine invariants using the property are constructed. Finally, the affine invariants are applied to the registration of Synthetic Aperture Radar (SAR) image and the effectiveness of the algorithm is verified. The simulation results show that the algorithm can reduce the disturbance from the speckle noise of SAR image and adapt the relatively large deviation of viewpoint.