Ultrahigh-repetition-rate frequency comb generation exhibits great potential in applications of optical waveform synthesis, direct comb spectroscopy, and high capacity telecommunications. Here we present the theoretical investigations of a filter-induced instability mechanism in passive driven fiber resonators with a wide range of cavity dispersion regimes. In this novel concept of modulation instability, coherent frequency combs are demonstrated numerically with rates up to sub-terahertz level. Floquet stability analysis based on the Ikeda map is utilized to understand the physical origin of the filter-induced instability. Comparison with the well-known Benjamin–Feir instability and parametric instability is performed, revealing the intrinsic distinction in the family of modulation instabilities. Our investigations might benefit the development of ultrahigh-repetition-rate frequency comb generation, providing an alternative method for the microresonators.