An environment adaptive face detection and recognition system is proposed in this paper to improve the recognition precision in traditional door access control systems. The two-stage convolutional neural network technology with precise division of labor and the cloud application technology based on TCP/IP is adopted to improve the performance of traditional systems. Experiment results, with the advanced MTCNN model, show that after 2 generations and 10 generations of training, it can’t reach the convergence, while the results are better after 20 generations of training. Practical tests demonstrate the potential applications in communities and companies where the number of occupants maintains is comparatively small due to the high precision and compatibility.