Fourier light field microscopy (FLFM) shows great potential in high-speed volumetric imaging of biodynamics. However, due to the inherent disadvantage of wide-field illumination, it suffers from intense background, arising from out of the depth-of-field signal and tissue scattered noise. The background will not only deteriorate the image contrast, making quantitative measurement difficult, but also introduce artifacts, especially in functional imaging of the neuronal network activity in vivo. Here, we propose the robust Fourier light field microscopy (RFLFM), which suppresses the background in FLFM by introducing structured illumination and computational reconstruction based on HiLo. The superior performance of RFLFM is verified by volumetric imaging of biological dynamics in larval zebrafish and mouse in vivo, at a volumetric imaging rate up to 33.3 Hz. The statistical results show that the fluorescence background can be significantly depressed, with the signal-to-background ratio improved by orders of magnitude and the whole image contrast improved by as much as ∼10.4 times. Moreover, we stress that, in functional imaging of neuronal network activity in turbid brain tissues, our system can avoid artifacts resulting from background fluctuations, while conventional light field microscopy fails. As a simple but powerful tool, we anticipate our technique to be widely adopted in robust, high-contrast, high-speed volumetric imaging.