In order to solve the problem that the blind source separation is difficult to be directly applied to the hyperspectral unmixing, the linear spectral mixture model was introduced in the presence of Abundance Non-negative Constraint (ANC) and Abundance Sum-to-one Constraint (ASC) as the objective function of the blind source separation to change the traditional independence assumption. Then, the Bat Algorithm (BA) was introduced to optimize the objective function. This algorithm solves the problem that the traditional gradient optimization algorithm is easy to fall into the local extremum. A method was proposed for dimensionality reduction, which is based on Singular Value Decomposition Denoising-orthogonal Subspace Projection (SVDD-OSP). The experimental results on synthetic data and real remote sensing data indicate that the proposed algorithm has a high convergence rate and a high accuracy. In addition, it has the strong anti noise interference ability and can be applied to the data with a low purity.