Shadows caused by tall buildings and trees and color distortion in visible image make that traditional color quantization cannot accurately describe the spectral difference of different objects on the ground. This defect declines classification accuracy finally. In view of shadows and color distortion problem in the lowquality visible image, we put forward an improved method. On the first stage of our algorithm, the problem of missing spectrum information caused by shadows is solved through sampling, analysing, and extracting shaded regions by double-threshold method and then classifying different kinds of shaded regions by object-oriented method. On the second stage, we obtain accurate areas of trees by fusing discriminative information [lidar intensity, digital surface model (DSM)] in order to compensate incomplete extraction caused by color distortion. The experimental results in comparison with ground truth obtained by manual work show that the classification accuracy is improved obviously compared with the results obtained by traditional Dempster-Shafer (D-S) fusing method.