In the smoothing process of image,it is very important for nonlinear filter to reserve fringe detail of image while filtering noise.A new adaptive median filter algorithm based on extreme value is presented in the paper. According to whether gray value of arbitrary pixel is its neighborhood extremum, all pixels are divided into doubtful noise pixels and signal pixels. The gray value of signal point is unchanged. Following the average variance of the nine multiscale and multidirectional windows including eight one dimensional windows and one two-dimension window, the gray value of noise point is selected adaptively. Test result indicates that new algorithm is superior to multistage median filter in performance of filter noise and detail preservation. At the same time, operating speed of the algorithm is faster than classical median filter.