The position coordinates of the observers' eyes were derived in real time based on the depth camera and human eye detection model. Reasonable display area was calculated based on coordinate transformation model. The ambient color temperature and brightness were estimated based on the color component data of the color sensor. Finally, the display state was adjusted to make it nearest to the color performance of ambient light based on the pre-calibrated display parameter mapping table. A scene fusion experimental system was built, which can be divided into five parts:position monitoring camera, image capture camera, color sensor, display device and processor. Fusion experiments were carried out in indoor scenes with color temperature of 6 354 K and illumination of 160 lx, and outdoor scenes with color temperature of 6 197 K and illumination of 848 lx. The experimental results show that this scene fusion scheme can adjust display area for the observers in different locations, and change display parameters according to ambient light information, then achieve excellent fusion effects. The single execution only takes 283 ms.