To solve the problem of visual unnaturalness caused by contrast-limited adaptive histogram equalization (CLAHE) forced blocking, this study proposes an infrared image enhancement method based on semantic segmentation. The semantic segmentation network segments the entire infrared image into category blocks instead of traditional rectangular image blocks. Each category block is individually subjected to contrast-limited histogram equalization to reduce over-enhancement. Finally, a new edge transition method is introduced to avoid abruptness between category blocks. The experimental results show that the proposed image enhancement method outperforms other contrast algorithms in terms of contrast and entropy and avoids the visual unnaturalness of traditional CLAHE with better visual effects.