In autonomous driving applications, the vision-based relative pose estimation is one of the core technologies to realize self-positioning of driverless cars. Aiming at the problems that the monocular vision system has a small field of view and the binocular vision system needs a overlapping field of view, a method for decoupling relative pose estimation of multi-camera systems with non-overlapping field of view is proposed. Use the different characteristics of the near and far points in the scene to decouple the rotation matrix and the translation vector. The relative rotation matrix is solved by the histogram voting method for the feature points of the distant scene, and the relative translation is solved by the sampling height method for the feature points of the near scene under the condition that the relative rotation matrix is known. The feasibility, robustness and accuracy of the proposed method are verified through simulation and actual experiment. Experimental results show that the relative attitude accuracy of the proposed method is better than 0.05°, and the relative translation vector direction accuracy is better than 2.5°, which has higher accuracy and better robustness.