Mode instability refers to an abruptly mode change when the average output power increases above certain threshold power, which results in degrading of beam quality. Based on the mode-coupled model, the threshold power of onset mode instability, which is induced by quantum noise seeding or intensity noise seeding, is investigated and analyzed in detail. Analysis results show that the threshold power can be improved by 2~5 times for adopting tandem pumping technology and partially doping technology. Increasing the initial signal laser power, suppressing the intensity noise of the signal laser and reducing the power launched into the high order mode can also improve the threshold power. The investigation provides a reference to the design of high-power fiber laser systems.