Aiming at the algorithm of traditional total variation image denoising and restoration based on the image gradient as its edge indicator, which cannot effectively distinguish between edges and ramps and remove the isolated noise in flat regions, a new edge indicator based on the quadratic differential in the local coordinate system is proposed to improve the traditional model. The improved model can adaptively select the most appropriate denoising scheme based on the edge indicator information of each pixel, and its regularization term and fidelity term can be weighted by the image fine information. And in the aspect of the numerical implementation, a new method based on the variation of the image gradient direction is adopted to discrete the divergence term to better preserve fine details effectively. Numerical experiments demonstrate that the new algorithm is superior to the traditional ones in the aspect of avoiding the staircase effect in ramp regions and preserving fine details.