A new type of optical fiber refractive index sensor was reported. The partial multimode fiber of Singlemode-Multimode Singlemode (SMS) fiber structures were side polished, and then the refractive index sensing properties were analyzed. The results show that multiple loss peaks can be observed in the spectrum of the SMS fiber structures because of the mutual interference by the high order modes of multimode fiber. The spectrum of the SMS fiber structures just drift a few nanometres when the SMS fiber structures are polished to close to the core; but when being side polished to the core of the fiber, the spectrum changes dramatically. After being side polished, the refractive index resolution of the SMS fiber structures will be higher. When the polishing depth is 10 μm and the polishing length is 10mm, the refractive index sensitivity of the SMS fiber structure is 703.2 nm/RIU. When the polishing depth is 15 μm, the refractive index sensitivity of the SMS fiber structure is 798 nm/RIU. When the polishing depth is 20 μm, the refractive index sensitivity of the SMS fiber structure is up to 1 190 nm/URI. A new type of optical fiber refractive index sensor can be realized with a single side polished SMS fiber structure, and the sensitivity of the sensor can be improved by increasing the depth of polishing area.