The polarization dependent gain(PDG) of fiber Raman amplifiers(FRAs) with bidirectional pumps was investigated systematically for the first time, by numerical method based on the nonlinear coupled equations describing the properties of fiber Raman amplifiers. The FRA with four pumps has 14 structures. First of all, the general PDG characteristics of the 14 different FRAs were investigated, and then the PDG characteristics of a specific structure were analyzed in detail, including the PDG induced by pump-signal Raman interaction, by pump-pump Raman interaction and by signal-signal Raman interaction. The results show that(1) the more backward pumps generates the smaller mean PDG values under constant number of pumps,(2) when the total number of pumps and the number of backward pumps are both constant, the backward pump wavelength has little effect on the PDG, and (3) the PDG of a bidirectionally pumped FRA is mainly determined by pump-signal Raman interaction. These results are useful and important for reducing the PDG of an FRA.