The current approach for binocular camera calibration is to obtain the rotation and translation relationship between cameras through matrix transformation, and the final parameters are obtained by optimization. The nonlinear optimization procedure is tedious, and the internal and external parameters of the camera are coupled with the lens distortion. We present a calibration method for the binocular camera with the combination of a distortion correction and a plane homography matrix. Firstly, based on the principle that three-dimensional linear projection to the image plane is still a straight line for the image without distortion, while any two points on the right line of vectors has the same direction, the direction of the vector angle should be zero. The distortion coefficient can be solved based on this characteristics. Then the measurement matrix is constructed by plane homography matrix, and the initial values of inside and outside parameters of the camera are obtained through matrix decomposition. Finally, all parameters of binocular camera are calculated by the nonlinear optimization. Simulations and experimental results show that this method is highly stable, and has equal accuracy compared to the traditional calibration methods. It spends less time and has better efficiency than the traditional calibration methods.