The reflecting matrix,transmitting matrix and internal radiation source function vector of the rough sea surface were deduced in detail,and were applied to the vector radiative transfer numerical model of the coupled ocean-atmosphere system named PCOART. The model was validated by the two rough sea-surface radiative transfer problems,one was the vector radiative transfer problem of the Rayleigh atmosphere,and the other was the radiative transfer problem of the coupled ocean-atmosphere system. The result shows that the model gives accurate simulation for the two aspects,especially with the relative error of Rayleigh scattering radiance less than 0.5%. Compared with the existing scalar radiative transfer numerical model of the coupled ocean-atmosphere system,this model can derive the radiance field and the Stokes vector field for the whole ocean-atmosphere system.