For English sounds with a large number of characteristic data and complex pronunciation variations, there are more problems in hidden markov models (HMM) than words, such as complexity calculation of wittby algorithm and probability distribution in gaussian mixture model.In order to realize the speech-independent English speech recognition system based on HMM and clustering, a combination of segmented mean algorithm, clustering crossover grouping algorithm and HMM grouping algorithm is proposed to reduce the dimension of speech feature parameters.Experimental results show that compared with the single HMM model, the algorithm not only improves the recognition rate of English speech 3%, but also improves the recognition speed of the system 20.1%.