A novel optimization-based method for designing wavelet filter banks in image fusion is proposed. The filter bank design is formulated as a nonlinear optimization problem. The objective function of the optimization problem consists of both the performance metrics of the image fusion, such as the root mean square error (RMSE), and those of individual filters. The optimization problem is solved using simulating annealing.