A novel selective ensemble learning algorithm is proposed based on Moore-Penrose pseudo-inverse. After individual learning machines had been trained independently, a set of initial weights were assigned to the individuals. Then an method based on Moore-Penrose pseudo-inverse was applied to optimize these weights for a minimum generalization error, and only those with high weights would be chosen for the final ensemble. Experiments show that this method is efficient in the computation complexity and easy to be implemented. Moreover, the method can achieve higher generalization ability with a much smaller ensemble size than other popular ensemble approaches.