When the target is several miles away from the infrared fisheye system, it will be a point target in the infrared image, so there is no target information of distance, geometry and texture, without which it is hard to assess the threat of target accurately. So the multi-target threat assessment of infrared fisheye system is studied. A multi-target threat assessment model of the infrared fisheye system is proposed. In the model, the distance and the radial velocity of each hour are derived from the initial distance taken by laser range finder, and hence the multi-target threat assessment model is established including the threat factors of target distance, radial velocity, course angle and angular altitude. Then considering the nonlinear characteristic of multi-target threat assessment, the radial basis function (RBF) neural network is used to solve the problem for its good self-adaptive and self study ability to solve nonlinear complex problems and the training sample generation is also discussed. After simulation experiment, it is found that this method is feasible and effective.