The outliers in data space can benefit the boundary decision procedure in the training of classifiers. A new outlier detection method based on non-parametric affinity propagation clustering in independent subspace analysis (ISA) feature space is proposed. Similar to support vectors in support vector machine (SVM), the outliers are used for the training of kernel-based classifiers. Proposed method analyzes the sample distribution in original unlabeled data space, of which the samples far from exemplars are selected as outliers, and also serves as high quality training set in both image classification and retrieval tasks. Experiments show the proposed kernel method via outliers detection outperforms state-of-art feature models. Also, the results validate the outliers detection promote classification accuracy indeed.