A method for defect detection and area calculation of exterior windows of buildings is proposed by combining infrared thermal imaging technology and image processing technology. Using equipment for detection of building exterior window defects, the differential-pressure method was utilized to detect the air penetration of an exterior window, and the defective area of the air penetration of this window was calculated. Infrared images of the exterior window of the building collected by an infrared thermal imager were subjected to image preprocessing, exterior window defect detection, and area calculation after inspection. Then, an infrared-image detection model of exterior window defects was established. The results show that preprocessing can make use of the weighted average method for grayscale processing, the median filter for noise reduction, image sharpening, and histogram equalization for image enhancement processing. The outcome of the aforementioned approaches is evident. The detection of the pretreatment infrared image, which is obtained using the Roberts algorithm, minimizes the difference between the test and experimental values. This makes the detection information closer to the actual position of the defect. A primary assessment of the airtightness performance level of exterior windows can be achieved by comparing the results provided by the proposed infrared image processing technology with airtightness on-site tests.