The simulation of the imaging system is very important to the demonstration, design and performance forecast of the optical remote sensing detector. As the remote sensing object, characterization and modelling of the scene will affect the simulation results directly. Aiming at the crucial geometry, spectrum and radiance characters, a digital scene modelling method based on the optical field theory was proposed. Using the plenoptic function comprised of geometrical coordinates, direction, spectrum and intensity, irradiance field onto the scene surface was modelled including the direct solar radiance, skylight radiance and reflected background radiance. With directional reflectance character of the surface, radiance field leaving the scene surface was also modelled. A simple small 3D scene was created to validate these models. The proposed method will provide digital scene model with multiple characters for the imaging system simulation and the new-style detector development.