In the cost aggregation methods based on tree structure, the weight support region is selected only by color information, and therefore it is easy to produce mismatching problem in the image boundary area. A variable weight cost aggregation algorithm for stereo matching based on horizontal tree structure is proposed to solve the problem. The initial disparity value is obtained by the cost aggregation of horizontal tree, the horizontal tree is reconstructed with initial disparity value and color information, and the final disparity map is obtained on the updated tree structure by cost aggregation. In the disparity refinement stage, an improved non-local disparity refinement algorithm is proposed with the pixel points that do not satisfy left-right consistency constraint introduced into the matching cost volume, which improves the matching accuracy of final disparity map. Performance evaluation experiments on all 31 Middlebury stereo pairs demonstrate that the proposed algorithm achieves an average error matching rate of 6.96% in the non-occluded areas without disparity refinement, and the cost aggregation takes 1.52 s on average.