Deep Residual Learning for Image Recognition (ResNet, 2015): skip connections make 100+-layer networks trainable

He, Zhang, Ren and Sun (Microsoft Research) show that letting layers learn residuals around identity shortcuts removes the degradation problem, training 152-layer networks that won ImageNet and COCO 2015. Residual connections are now in essentially every deep model, Transformers included. 12 pages.

Companies in this report

MicrosoftHugging FaceGitHub Copilot