Attention is All You Need

This paper introduces the Transformer, a novel neural network architecture based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. It demonstrates that this architecture can be trained more efficiently and achieve state-of-the-art results in machine translation tasks, suggesting that attention mechanisms are sufficient for modeling sequences and can outperform recurrent and convolutional networks.