To evaluate the security of a chaos optical communication system employing the polarization-shift keying (PolSK) modulation technology, its chaos characteristic needs to be verified. In this paper, an analysis was done for the signal of this system. Three methods were used to judge whether the signal was maintaining chaos characteristics or not: watching the strange attractor in three-dimensional phase space, computing the largest Lyapunov exponent by the equation which meets and Wolf’s method, and evaluating the self-power spectrum density function. As a result, the strange attractor was clearly watched, the largest Lyapunov exponent was positive 0.0364 and 0.0106 respectively, and the self-power spectrum was wide and continuous with the noise background. The evaluation of chaos for the signal transmitted in the system is therefore presented. On the other hand, the minimal embodied dimension of the signal was given by the false nearest neighbors (FNN) method and it reached 6, which showed the higher dimension chaos characteristics of the system. Adding the analysis of the ability of anti-attack for the system, it is concluded that the system has higher security than the normal chaos masking schemes.