An all-fiber high-power linearly polarized chirped pulse amplification (CPA) system is experimentally demonstrated. Through stretching the pulse duration to a full width of approximately 2 ns with two cascaded chirped fiber Bragg gratings (CFBGs), a maximum average output power of 612 W is achieved from a high-gain Yb-doped fiber that has a core diameter of 20 μm with a slope efficiency of approximately 68% at the repetition rate of 80 MHz. At the maximum output power, the polarization degree is 92.5% and the M² factor of the output beam quality is approximately 1.29; the slight performance degradations are attributed to the thermal effects in the main amplifier. By optimizing the B-integral of the amplifier and finely adjusting the higher-order dispersion of one of the CFBGs, the pulse width is compressed to 863 fs at the highest power with a compression efficiency of 72%, corresponding to a maximum compressed average power of 440.6 W, single pulse energy of 5.5 μJ and peak power of about 4.67 MW. To the best of our knowledge, this is the highest average power of a femtosecond laser directly generated from an all-fiber linearly polarized CPA system.