The adaptive tracking algorithm for the aerial small target is proposed based on the multi-domain convolutional neural networks (MDNet)and the autoregression (AR) model,to solve the tracking drift problem that the pseudo targets and the small target converge in the sky background. Firstly, the positive samples of the first frame in the image sequence are collected by MDNet to train the bounding-box regression model. Secondly, the AR model with its order and parameters determined by the Akaike information criterion and least squares method is trained to estimate the target track and to predict the target position. Finally, the region of sampling candidate is constrained since MDNet collects samples centered on the predicted target location, and then the target position is adjusted by the bounding-box regression model. Eight groups of benchmark video sequences are tested with the proposed algorithm and another seven classical tracking algorithms, and obtained results are compared. The experimental results show that the success rate and the average overlap rate of the proposed adaptive tracking algorithm are higher than those of other algorithms, and the proposed algorithm has higher accuracy and robustness.