In order to improve the vision navigation of unmanned aerial vehicle more real-timely and accurately, a pose estimation method step by step based on monocular vision is proposed using rectangular geometry feature and coplanar feature of points and lines. This proposed method shoots the rectangular target under any position, and two orthogonal vanishing points are achieved. The constraint equation of projection between infinite points and orthogonal vanishing points are established to calculate the rotation parameters. And also a constraint equation is established by using the coplanar feature of camera optical, rectangular target and its projection, which are used to solve the normal vector of rectangular target under camera coordinate system. Based on the invariance feature of linear transformation under Euclidean space, the four vertices of rectangular target under camera coordinate system can be linearly calculated and the solution of the translation estimation can be accomplished through the translation between camera and world coordinate system. Aiming at restraining the influence of noise to estimation, the indicated function based on the coplanar feature of points and lines is obtained, using the NM optimization algorithm to realize the nonlinear optimization of pose parameters. Experimental results show that the proposed method has the advantages of high precision, real-time and wide application. The robustness of estimation algorithm is improved by reducing the image noise through parameter optimization.