ing at the problems of insufficient usage of context information and unclear image edge segmentation in image semantic segmentation, a network model based on multi-scale feature extraction and fully connected conditional random fields is proposed. RGB and depth images are input into the network in a multi-scale form, and their features are extracted by a Convolutional neural network. Depth information is added to supplement the RGB feature map and obtain a rough semantic segmentation, which is optimized by the fully connected conditional random fields. Finally, fine semantic segmentation results are obtained. This proposed method improves the precision of semantic segmentation and optimizes the image edge segmentation, which has a practical application.