Aiming at the unadaptive problem of classical Mean Shift algorithm on changing target shapes in tracking process, in this paper, a tracking window adaptive Mean Shift target tracking algorithm is proposed. This algorithm uses the moment characteristic to derive out an evaluate approach of target true area. By the true target area, the covariance matrix of image characteristics in target window is corrected, and the length and orientation of principal axis are figured out. Test results indicate that this algorithm retain the simple and validity of the classical Mean Shift algorithm, at the same time the tracking window of the proposed approach can adaptive the rotation and scale changing in target tracking, and also having robust to a certain degree.