Side channel effects such as temporal disparity and intensity fluctuation of the photon pulses caused by random bit generation with multiple laser diodes in high-speed polarization-based BB84 quantum key distribution (QKD) systems can be eliminated by increasing the DC bias current condition. However, background photons caused by the spontaneous emission process under high DC bias current degrade the performance of QKD systems. In this study, we investigated the effects of spontaneously emitted photons on the system performance in a high-speed QKD system at a clock rate of 400 MHz. Also, we show further improvements in the system performance without side channel effects by utilizing the temporal filtering technique with real-time field-programmable gate array signal processing.