To address the problems of low accuracy, slow convergence, and poor stability in using the back-propagation (BP) neural network for inverse modeling of dual band-notched ultra-wideband filters, this paper proposes an approach to optimizing inverse modeling based on the BP neural network with an improved ant lion optimization (IALO) algorithm and the Huber function. This method improves the ant lion optimization algorithm by serializing the boundary contraction factor, introducing dynamic update coefficients, and adding the Cauchy mutation. Then, the IALO algorithm is applied to optimize the weights of the forward model and thereby speed up the modeling. Subsequently, the Huber function is used to evaluate the neural network. The accuracy and stability of the model are thus improved. This method is used for a double band-notched ultra-wideband filter. Experimental results show that compared with BP inverse modeling, the proposed method reduces the length, width, and frequency mean square errors by 97.44%, 99.43%, and 96.15%, respectively, and shortens the average running time by 66.01%. The multi-solution problem of inverse modeling is solved, and the speed and accuracy of filter design are improved.