Background noise is one of the most important properties for the interferometric fiber optic vector sensor systems, which limits the systems’ application of the detection of the long distance and low frequency targets. To reduce the background noise, a noise suppression scheme based on an adaptive noise canceller is proposed. A pressure insensitive reference fiber optic Michelson interferometer is added as a reference sensor, whose structural parameters are the same as those of the sensing fiber optic interferometers. Then the background noises of the pressure or acceleration signal, which are highly correlated with the reference signal, can be cancelled by the normalized least root mean square error algorithm. The results of a lake trial show that the scheme can effectively reduce both the background noise of the pressure and the accelerometer channels, which can suppress the 50 Hz multiples of the electromagnetic interference by 15～25 dB, and the flat phase noise above 500 Hz by about 3 dB.