Abstract:A remote control method for mechanical system based on head mounted device and operating handle, which using the pose information of operating handle to control the moving of the mechanical system , and calculating the relative pose between head mounted device and operating handle to control the image acquisition equipment on the mechanical system. Based on this method, a system was developed on an AR glasses engineering prototype which connected to a smart phone, and the feasibility is verified by experiments on two types of mechanical systems: a quadruped robot and a four-wheel car. The experimental results show that this method is feasible and practical, and makes the user feel more immersive. At the same time, there is no need to deploy external attitude capture equipment, and there is no requirement for environment, so it can be used in indoor and outdoor environments.