We propose an automatic three-dimensional (3D) pupil tracking backlight system for holographic 3D display system with large image size and full-parallax accommodation effect. The proposed tracking module is applied to a holographic 3D display system with two sets of directional holographic imaging module composed of 2×2 large scale lens array and 22-inch high-resolution liquid crystal display 3D panel. System architecture is described and experimental results are presented.