The advance of visual tracking has provided unmanned aerial vehicle (UAV) with the intriguing capability for various practical applications. With promising performance and efficiency, discriminative correlation filter (DCF)-based trackers have drawn great attention and undergone remarkable progress. However, the boundary effect and filter degradation remain two challenging problems. In this work, we propose a novel Adaptive Spatio-Temporal Regularized Correlation Filter (ASTR-CF) model to address these two problems. The ASTR-CF can optimize the spatial regularization weight and the temporal regularization weight simultaneously. Meanwhile, the proposed model can be effectively optimized based on the alternating direction method of multipliers (ADMM), where each subproblem has a closed-form solution. Experimental results on DTB70 and UAV123@10fps benchmarks have proven the superiority of our method compared to the state-of-the-art trackers in terms of both accuracy and computational speed.