A fast image defogging algorithm based on edge-maximum filter was proposed to address halo effect and color distortion caused by the existing defogging methods. Firstly, an edge-maximum filter was used to recover the undervalued dark pixels obtained by edge detection, which was to receive an accurate transmission map and eliminate the halo effect. Then in order to gain a high contrast dehazing image, all the dark pixels were multiplied by a scaling factor to improve the dynamic ranges of the transmission. Finally, two brightness thresholds and one flat threshold were set to eliminate the influence of high light objects in the image and obtain a more accurate airlight, which keeps a high color fidelity in the dehazing image. The simulation results show that the proposed method, compared with other algorithms, could eliminate the halo effect and achieve the dehazing image with high contrast and high color fidelity, especially for the images containing high light objects or rich details. Meanwhile, the computational speed is also improved.