When the eye tracker is collecting infrared eye movement data, due to the rapid movement of the subject’s eyeballs or the inability to keep relatively still with the instrument, some of the collected eye area images are defocused and blurred. This paper proposes a semantic segmentation optimization system, which is called super real-time semantic segmentation network (S-RITnet). First, a pixel-level annotation data set with a 4∶1∶1 ratio of images in the training set, validation set, and test set is created. Then, the enhance super-resolution generative adversarial network and contrast-limited adaptive histogram enhancement algorithm are used to repair the blurred eye area data set image. Finally, based on real-time semantic segmentation net and the autonomous data set (including the repair data set), perform network training to realize the semantic segmentation of the eye area image and evaluate the obtained segmentation module. The experimental results show that the optimization scheme can effectively optimize the quality of eye area images. Compared with the low-quality eye images training module, the mean intersection over union and F1-score evaluation of S-RITnet increased by 0.0247 and 0.024 respectively.