Ultra-short linear cavity is the simplest configuration to achieve stable single-frequency laser output. In order to increase the rare-earth doping concentration we choose phosphate glass as the laser gain medium. The spectroscopic properties of heavily Yb3+-doped phosphate glasses(P2O5-B2O3-K2O-BaO-Al2O3-Nb2O5-Sb2O3) have been investigated for developing 1064 nm single-frequency fiber laser. The experimental results show that the molar fraction solubility of Yb2O3 in this phosphate glass system is up to 6% (mass fraction 15.5%). Based on the reaction atmosphere process,the OH- groups can be removed effectively and the fluorescence lifetime of the 2F5/2 state of Yb3+ ion in phosphate glass reaches 1.84 ms. The Yb3+-doped phosphate glass fiber has been drawn by the rod-in-tube technique. A continuous-wave single frequency laser with 51.6 mW laser output at 1063.5 nm is achieved in a 1.4 cm phosphate glass fiber through a narrow bandwidth Bragg grating.