For large aperture space telescope with segmented primary mirror, it is necessary to study in-orbit cophasing error correction method. The image sharpness function is taken as metric function. The relationships between co- phasing errors of segmented mirror and the metric function are analyzed by simulation under monochromatic illumination or white- light illumination. Simulation results show that under monochromatic illumination, the dynamic range of piston error correction is less than 0.5 λ. Under white-light illumination, global optimization algorithm should be adopted because of the local maximums of metric function. When piston error is greater than 0.5 λ, stochastic parallel gradient descent (SPGD) algorithm is prone to falling into local maximum and genetic algorithm is capable of finding out the global optimum but with relatively slow iteration speed. A hybrid optimization algorithm based on SPGD algorithm and genetic algorithm is proposed for co-phasing error correction of segmented mirror. When piston error is between - λto λ, the correction accuracy of the hybrid optimization algorithm can be better than 10 nm.