The noise equivalent temperature difference (NETD) of infrared cameras is a crucial criterion for evaluating the imaging quality. The position and aperture of the blackbody source, target, and entrance pupil (the collimator and the camera to be measured as a whole) in an infrared camera NETD test device should meet the specific mathematical relationship, otherwise the test accuracy will be reduced. Therefore, based on geometrical optics theory and radiology, the spatial distribution of the blackbody source, target, and entrance pupil in the infrared camera testing system is deduced in this study. Additionally, the relationship between the measured temperature at the target hole and the blackbody source temperature at the rear is explored. A mathematical model has been established to guide the test of the infrared camera test system. Combined with a test system, the relative error is verified to be less than 7.4% when the spatial distribution of the test device meets the proposed model requirement; otherwise the test accuracy decreases rapidly. The temperature uniformity and stability at the target hole are analyzed when the target is used.