In the characterization of elastic properties of tissue using dynamic optical coherence elastography, shear/surface waves are propagated and tracked in order to estimate speed and Young's modulus. However, for dispersive tissues, the displacement pulse is highly damped and distorted during propagation, diminishing the effectiveness of peak tracking approaches, and leading to biased estimates of wave speed. Further, plane wave propagation is sometimes assumed, which contributes to estimation errors. Therefore, we invert a wave propagation model that incorporates propagation, decay, and distortion of pulses in a dispersive media in order to accurately estimate its elastic and viscous components. The model uses a general first-order approximation of dispersion, avoiding the use of any particular rheological model of tissue. Experiments are conducted in elastic and viscoelastic tissue-mimicking phantoms by producing a Gaussian push using acoustic radiation force excitation and measuring the wave propagation using a Fourier domain optical coherence tomography system. Results confirmed the effectiveness of the inversion method in estimating viscoelastic parameters in both the viscoelastic and elastic phantoms when compared to mechanical measurements. Finally, the viscoelastic characterization of a fresh porcine cornea was conducted. Preliminary results validate this approach when compared to other methods.