Phase diversity (PD) can not only be used as wave-front sensor but also as image restoration technique. However, its computations have been perceived as being too burdensome and it is not to satisfy the real-time requirement. On the principle of minimum data communication between CPU and GPU, we analyze the process of PD algorithm and the experiment result shows that the part of optimization is appropriate to be realized on GPU. Firstly, the process of PD is analyzed and then task partition is implemented on dual GPUs. Under the experiment condition of wavefront aberration of PV=2λ, after 50 iterations, the execution time of single GPU and dual GPUs are 53 ms and 45 ms respectively for image size of 256 pixels×256 pixels.