In the integral imaging light field display, the introduction of a diffractive optical element (DOE) can solve the problem of limited depth of field of the traditional lens. However, the strong aberration of the DOE significantly reduces the final display quality. Thus, herein, an end-to-end joint optimization method for optimizing DOE and aberration correction is proposed. The DOE model is established using thickness as the variable, and a deep learning network is built to preprocess the composite image loaded on the display panel. The simulation results show that the peak signal to noise ratio value of the optimized image increases by 8 dB, which confirms that the end-to-end joint optimization method can effectively reduce the aberration problem.