In complex background scene, the variation of brightness or contrast may affect the performance of image segmentation.To deal with this problem, a new algorithm for image segmentation was proposed based on phase information of geometric edges.The algorithm first extracted phase information of edge using Inter-Coefficient Product (ICP) transform, and obtained the image edge using the phase information.Then, according to the phase information and amplitude, a pixel was chosen as the seed of growing process, and the decision criterion was established.Finally, the whole image was segmented by using region growing.The simulation results show that:the proposed method is not influenced by the variation of brightness or contrast, and can implement infrared image segmentation accurately and effectively.The algorithm also helps to avoid missing the edge information of image.