An image segmentation algorithm based on selective enhancement is proposed to improve the occlusion recognition accuracy of shadows and leaves for photovoltaic (PV) arrays. The proposed algorithm enhances module images in the HSV color space to acquire “moss images”, to realize discoloration of shade and improve the contrast between shaded and unshaded parts. Then, the “moss images” are segmented to acquire the shaded outlines. Experimental results demonstrate that the position and areas of shadows and attached shelters (leaves, etc.) are more accurate after being processed by the proposed algorithm, which gets convenient conditions for the removal of shades and provides a basis for accurate modeling of PV arrays under partial shadow conditions.