After long time working, the response characteristics of infrared focal plane array (IRFPA) often drift slowly, and the image quality always degrades. Aiming at the problem, the characteristic of residual noise after two point correction (TPC) is analyzed. An adaptive non-uniformity correction algorithm is proposed based on the TPC algorithm. The images after TPC is decomposed by wavelet, and the Bayesian threshold is used to estimate signal and noise variance point by point, the residual nonuniformity noise is calculated and removed. Experimental results show that the proposed algorithm can remove the residual nonuniformity noise effectively and avoids the image degradation caused by the response drift of IRFPA.