Microlens array is used to record the information of space scenes in integral imaging system by only one imaging process. During the acquisition of the depth, the parameter setting and calibration of the camera is unnecessary, then the depth mesurement process is simplified, therefore the integral imaging has a great advantage. However, due to the low resolution of the elemental images collected in system, the stereo matching between the elemental images is difficult to reach and susceptible to the recorded noise. Based on the advantage of integral imaging system can calculateand reconstruct multiple viewing images, make full use of the redundant information provided by the multiple viewing images to improve the stereo matching. An adaptive window with color similarity is utilized in the stero matching between the reference viewing image and other viewing images to obtain the disparity information. The function fitting for the multiple disparities of the reference viewing image is performed by considering the property that the disparity is proportional to the sampling distance between this pair of viewing images. This method can eliminate the mismatch resulting from factors such as the recording noise and make the discrimination more accurate between the depth levels of the object.