In the automatic evaluation of grain size in metallographic tissue, the accuracy of grain boundary recognition directly affects the accuracy of assessing the grain size of metallographic tissue. To address the problems of dense grain boundaries, complex edges and low accuracy of grain boundary recognition in steel metallographic images, a lightweight U-net convolutional neural network-based grain boundary segmentation method is proposed, which splices shallow feature layers with jump connections in the upsampling process, so that the network learns more effective feature information; reduces the number of layers and adds a single convolutional feature extraction process, reducing the number of network parameters and improving the speed and accuracy of the prediction of grain boundaries. Experimental results show that the method achieves a pixel accuracy of 93.91%, a specificity of 96.73% and a sensitivity of 81.6% on a test set of 117 metallographic images. Compared with the conventional U-net network, the pixel accuracy is improved by 0.2% and the number of network parameters is relatively reduced by 61.5%. The method is effective and superior for metallographic grain boundary segmentation.