Face recognition under varying illumination is difficult, and we propose a new illumination normalization method based on inherent texture features within micro-neighborhood, which transforms the nonlinear process of grey level changes with illumination into a linear process in the micro-neighborhood. So, some harmful influence on image recovery by uncertainties including that from image structure is avoided. The inherent texture future is illumination insensitive, the structure of which is coded. The least squares estimation is used to estimate the relationship of grey level changes at certain illumination direction and the coded features, and illumination normalization is then performed according to the estimated relations. The computational complexity of this algorithm is low. The experimental results show that by implementing the algorithm in face recognition system, we can achieve an average recognition ratio of 94.1% on Yale B database with 90° illumination change.