Based on the Mie scattering theory, the scattering effect of related parameters of sea foam on photon polarization scattering is analyzed with the hollow layered particle model. The equations of quantum bit error rate and photon-polarization fidelity are deduced. The whole process of photons passing through a foam layer is simulated with the Monte Carlo method. The relationships among the quantum bit error rate, foam scattering coefficient, foam absorption coefficient, thickness of foam layer and scale parameter of foam particles are discussed. The simulation results show that the increasing of the foam scattering coefficient and the thickness of foam layer may result in the increasing of the scattering times, which strengthens the depolarization degree of photons and reduces the photon-polarization fidelity. When scattering coefficient of foam is over 0.5 cm-1 and the thickness of the foam layer is over 6 cm, the quantum bit error rate caused by the foam reaches to 6.5%. The foam absorption increases the transmission loss and decreases the quantum key generation rate. Within the classical value range of the foam scale parameter, the quantum bit error rate decreases with the increase of the scale parameter.