Passive terahertz imaging has been utilized in strategic locations for personnel security screening owing to its safety concealment and clothing penetration. Multipolarization imaging can increase the information dimension and enhance the system's performance. In this study, we comprehensively evaluate multipolarization image quality for passive terahertz imaging detection considering the challenges in selecting the polarization mode and the unclear characteristics of multipolarization images. First, a terahertz radiation brightness temperature model of the human body and hidden objects with arbitrary polarization was examined. Then, four linear polarization images were obtained using the polarization adjustable imaging system, and six polarization parameter images were computed and analyzed. Furthermore, multipolarization images were compared using the traditional global no-reference evaluation indexes. Finally, the receiver operating characteristic (ROC) curve and differential signal-to-noise ratio (DSNR) were employed to quantitatively estimate the local image quality used for detection. Observer subjective evaluation statistical tests were also conducted to confirm the reasonability of some global and local indexes. The results show the importance of reasonable polarization selection and multpolarization fusion in improving imaging detection performance.