To achieve displacement measurement using the imaging grayscale of the image sensor, it is necessary to use the camera to image the target during the displacement process, and then establish the mapping relationship between the displacement value and the imaging grayscale value, i.e., the imaging grayscale model, and the measurement accuracy of this method depends on the imaging grayscale model and the grayscale noise level. In the actual measurement process, external error sources such as uneven illumination, target manufacturing errors, aberrations of the camera imaging system, and the variability of imaging characteristics between different internal image sensor units can cause the imaging grayscale model to deviate from the ideal situation, thus affecting the measurement results. To further improve the measurement accuracy, the modeling error caused by the aforementioned nonlinearities are taken into account, and a class of imaging grayscale models combining Fourier series and higher-order polynomials is proposed to improve the generalization approximation ability of the models and thus the modeling accuracy, correcting the grayscale distortion caused by the aforementioned error sources. On this basis, the displacement is solved by the sequential solution method based on the displacement continuity principle, and the experimental results show that the standard deviation of the displacement measurement error under the 10.46 mm stroke using this improved model is reduced from 56.4 μm before correction to 1.5 μm.