Aiming to solve the problems of complex drug-grabbing tasks and low efficiency of existing pharmacists, a vision-based drug-grabbing robot system is designed, and a drug-identification algorithm based on the fusion of local feature matching and based on Mean Shift algorithm is proposed. The complete system realizes drug identification, positioning, and grabbing tasks. The drug-grabbing robot installed between the drug racks receives information of the medicine sent by the upper computer, identifies the medicine through its camera, matches the medicine on the transporter using the AKAZE algorithm, obtains the coordinates of the medicine, and grabs and places the matched medicine on the transporter. After the sorting robot located in the designated area for taking the medicine obtains information of the medicine sent by the upper computer, the dynamic medicine on the transporter is identified twice via the improved Mean Shift algorithm, and the identified medicine is grabbed and placed in the designated area for taking the medicine to complete the taking function of the whole system. Experiments show that the system can accurately identify the drugs with the guarantee of secondary sorting and can accurately locate drug information with a small error, which is suitable for drug grabbing in pharmacies.