Aiming at quantitative assessment of nonuniformity of infrared focal plane array (IRFPA), a scene-based assessment method is researched. First of all, distribution characteristics of remote sensing image sequence are analyzed by real remote data, an assessment method by constructing standard image based on sequence sorting and wavelet transform is proposed for small image sequence. Then, combined past assessment method for nonuniformity, response nonuniformity of constructed standard image is proposed. Finally the algorithm of construction standard image is verified by blackbody radiometric calibration method (BRCM) and statistical method separately. And this assessment method for nonuniformity is used in image sequence using different scene-based nonuniformity correction. The experimental results show that for the standard images constructed by two groups of image sequences of 128 frame, the nonuniformity calculated by new method is 0.92% less than that of BRCM, and 0.59% more than that of statistical method, respectively. This assessment method can assess the nonuniformity of infrared remote sensing image.