Aiming at the problem of object-based image retrieval, a new Multi-instance Learning (MIL) algorithm based on Fuzzy Support Vector Machine (FSVM), called FSVM-MIL algorithm is presented. The standard MIL problem assumes that a bag is labeled positive if at least one of its instances is positive, otherwise, the bag is negative. FSVM-MIL algorithm treats the whole image as a bag, the segmented regions as instances, and then each image with the desired object is labeled as a positive bag, while the other is labeled as negative bags. In order to address the ambiguity of instance labels in positive bags, the Diverse Density (DD) method was used to search multiple hypotheses, and according to noisy-or probabilistic model, a fuzzy membership function was defined which can give different fuzzy factors to these instances in the positive bags. Thus, the multi-instance learning problem was converted to a FSVM learning problem. Experimental results based on the SIVAL data set show that this algorithm is feasible, and the FSVM-MIL is competitive with other state-of-the-art multi-instance learning algorithms.