Three kinds of polymeric films with different rigidity, poly (methyl methacrylate) (PMMA), poly (N-vinylcarbazole) (PVK) and polystyrene (PS), which were doped with same concentration (mass fraction 0.003) of azo-dye 4'-[[2-(acryloyloxy)ethyl]ethylamino]-2-chloro-4-nitroazobenzene (disperse red 13, DR13), were prepared. The photoinduced birefringence effect of these three kinds of samples were studied by using a linearly polarized control beam from an Ar+ laser (514 nm) and a linearly polarized probe beam from a He-Ne laser (632.8 nm). It was found that under the same control beam power (15 mW) and with the same sample’s thickness, the PVK with large matrix rigidity showed small birefringence effect while polystyrene with small matrix rigidity showed large birefringence effect. Based on the four-energy-level model of photoisomerization of azo chromophores, a phenomenological model, in which a parameter s was introduced to describe the rigidity of polymer matrices, was presented to simulate the photoisomerization and orientation of the azo chromophores in polymeric matrices with different rigidity. The influence of the matrix rigidity parameter s on the photoinduced anisotropy of the azo-dye containing samples was investigated by numerical calculation. The computed results showed that large rigidity samples present small birefringence effect and the experimental results could be explained qualitatively by the theoretical simulation results.