An image fusion method was proposed based on the filter banks for merging a high-resolution panchromatic image with a low-resolution multispectral image. The panchromatic image was first decomposed into sub-images by using analysis filters and then the low-frequency sub-image of the panchromatic image was directly substituted by the multispectral image to merge a high-resolution panchromatic image with a low-resolution multispectral image. Finally, the fused image was reconstructed by using the synthesis filters. Experimental results indicate that the proposed approach can obtain the better trade-off between the spectral information of low- resolution multispectral sensor and the spatial properties of high-resolution panchromatic sensor by adjusting the channel number.