A novel measurement method for the torsional vibration of high-speed rotary machine is proposed, based on laser Doppler technique and the principle of optical heterodyne. The article analyzes its principle, worked out the mathematical model of the optical configuration, and tests the feasibility and accuracy respectively. The laser beams launch at two points of the same section of the rotating shaft. The reflected beams are shifted at frequency domain according to Doppler effect. The frequency shift is proportional to the rotating velocity. With the proper optical configuration, the beams of two successive times are heterodyned on the photo-detector, and the frequency of the photoelectric current is proportional to difference of velocity at the two times. By making the interval of the two successive times small enough so that the angular acceleration is measured directly. Compared with other measurement methods based on laser Doppler technique, the angular acceleration is measured directly, therefore the measurement of torsional vibration is more real-time and dynamic range is extended greatly. It is essential to the monitoring running state and failure diagnosis of large-scale gyration machine with high-rotational velocity.