Abstract: Rail defect detection is crucial for railway safety and reducing maintenance costs. Faced with the challenges posed by the expansion of the railway network and the scarcity of defect samples, particularly the issue of overfitting under few-shot conditions, this study proposes a few-shot deep learning algorithm for the detection of rail surface defects based on multi-scale WideResNet(Wide Residual Networks), which employs data augmentation techniques to expand the finite training set, thereby improving the model's generalization capability. By leveraging transfer learning strategies, it applies pre-trained deep learning models to rail defect detection tasks with extracting multi-scale features, reducing reliance on large annotated datasets. The study designs few-shot deep learning model strategies and constructing metric learning module that enable rapid learning from limited annotated data and effective generalization. Experimental results demonstrate that the algorithm can effectively detect rail surface defects under 10-shot conditions, achieving a model precision mAP of 83.6% and a recall rate as high as 93.8%. This research provides new technology for the automated detection of rail surface defects, which is of great significance for enhancing the safety and economy of railway transportation.