The feature location noise from feature-based homography estimation methods is isotropic and non-identical distribution, and it effects the accuracy and robustness of homography estimation methods significantly in practical applications. Therefore, this paper proposes a high accuracy and robust homography estimation method based on location noise of feature points. The method uses a covariance matrix to characterize the location noise of feature points and takes an inner point verification method based on Covariance matrix Weight SAmple Consensus(CWSAC) to improve the robustness of the homography estimation method. Finally, a high accuracy homography matrix refined method, Covariance matrix Weight Levenberg-Marquardt( CW L-M) is proposed by combining covariance matrix with Levenberg-Marquardt method, and it improves the estimation accuracy of homography matrix by redefining a optimized object function. The experiments on simulation data and real images show that as compared with state-of-the-art methods, such as RANSAC(RANdom SAmple Consensus),LMedS(Least Median of Squares), PROSAC(PROgressive SAmple Consensus), M-SAC(M-estimator SAmple Consensus)and MLESAC(Maximum Likelihood SAmple Consensus), the accuracy of homography estimation has improved greatly and the root mean squares of reproject error has reduced 3%- 21% than that of the subprime method in the same location noise and the same inlier proportion. In addition, the proposed method is robust to the noise level and inlier proportion changing.