Aiming to infrared and visible image fusion effect evaluating question, an assessment method bonding structural similarity and region of interest was proposed based on structural similarity assessment method and human visual characteristics. The method divided region of interest and residual region from infrared and visible image respectively based on different image-forming feature,which indicate the different image characteristics. Then the different weight coefficients were endowed to different image regions based on the visual sensitive level. Compared with the prevenient evaluating method, the proposed method needs no standard reference image participation and upgrades the weight value of image important characteristics in assessment result. Experiment results indicate that the proposed methods is more consistent with subjective evaluation results.