Three-dimensional (3D) display has attracted widespread attention all over the world for its intuitive and natural performance. At the same time, three-dimensional interactive technology is also developed fast. A real-time interactive system based on infrared structured light and single-camera is carried out. Infrared structured light is projected onto interactive gesture as encoded procedure, and then infrared images of user′s hand are captured by the camera in real time. An image processing algorithm implemented by OpenCV compares the difference between two consecutive images to obtain the movement information of structured light on user′s hand. Therefore, user′s interactive gestures including movement in three dimensions, zooming and rotating, can be recognized and then transported to three-dimensional display devices in real time. A prototype-equipment is proposed using off-the-shelf CMOS camera and infrared LED. Experimental results show that three-dimensional freehanded interaction can be realized by this approach in low cost.