For judging the security of the chaos optical communication system employing the polarization-shift keying (PolSK) modulation technology, its chaos characteristic needs to be verified above all. So analysis is done for the signal of this system. Three methods are used to judge the signal maintaining chaos characteristics or not: watching the strange attractor in three-dimensional phase space, computing the largest Lyapunov exponent by the equation it meets and wolf method, and computing self-power spectrum density function. As a result, the strange attractor is clearly watched, the largest Lyapunov component is positive 0.0364 and 0.0106, respectively, and the self-power spectrum is wide and continuous with the noise background. Therefore, the judgment of chaos for the signal transmitted in the system is presented. On the other hand, the minimal embodied dimension of the signal is given by the false nearest neighbours (FNN) method and it reaches 6, which shows the higher dimension chaos characteristics of the system. Adding the analysis of the ability of anti-attack for the system, it is concluded that the system owns higher security than the normal chaos masking schemes.