Method of adaptive windowed Fourier transform based on Hilbert-Huang transform (HHT) is presented to address the issue of adaptive selection of window size and spectrum aliasing in multiscale windowed Fourier transform for phase retrieval of fringe pattern. Spectrum obtuined by HHT is analyzed to adaptively determine the background and instantaneous frequencies which can describe the change of signals in detail for fringe pattern. By using the presented steps, local stationary regions are located adaptively according to instantaneous frequencies, and then scale factors of window function are got. Before performing adaptive windowed Fourier transform, the background which has been determined is eliminated from fringe pattern to avoid spectrum aliasing caused by expansion of zero-spectrum especially without extra computation. Compared with the method such as the ridge of wavelet transform or the ridge of S transform, the presented method is not limited by hypothesis of linear approximation and changing slowly for measured phase. Experments illustrate the effectiveness and feasibility of this method which is accurate and robust even for measurement of object which carries with steep edge or complex surface.