An active visual system for intelligent tracking of moving object based on Digital Signal Processing (DSP) was designed with Expectation Maximization (EM) algorithm and a novel zonal logical strategy for camera pan/tilt control was presented. The dada set of the moving object gray feature was treated as Gaussian mixture model which included gray pixel position，and the expectation was employed as objective function. The localization and shape size of object in every frame which could be achieved during the estimation process with the EM algorithm were regarded as auto-control signal of the camera. The experimental results by using our system with several real image data demonstrate that the proposed scheme is not only practical and easy to perform the moving object intelligent tracking, but also robust for tracking. The system test shows that the processing for 320 pixels ×240 pixels image achieves 20 frame/s.