The CamShift algorithm is a typical method of moving image tracking, but the accuracy of CamShift is low and it is easy to lose the target. In order to improve the accuracy, this paper applies the Gauss hybrid model to the target tracking operation in the CamShift algorithm. First the Gauss mixture model marked the local area, and the CamShift as the initial search window, improve efficiency; then track the target when the target area center CamShift algorithm with window center difference method to calculate the comparison, determine the subsequent search window frame, avoid the loss of target tracking. Finally, experiments show that this method can track the target effectively, and still has very high accuracy in the case that the color difference between the target and the background is small.