# Attention Is All You Need

Implementation of [Attention Is All You Need](https://arxiv.org/abs/1706.03762) (Vaswani et al., 2017).

## What this implements

The Transformer architecture — an encoder-decoder model based entirely on multi-head self-attention mechanisms, eliminating recurrence and convolutions. This implementation covers the full architecture as described in §3 of the paper: scaled dot-product attention, multi-head attention, positional encoding, the encoder and decoder stacks, and the output projection with weight tying.

## Quick start

```bash
pip install -r requirements.txt
```

```python
from src.model import Transformer, TransformerConfig

config = TransformerConfig()
model = Transformer(config)

# Example: encoder-decoder forward pass
import torch
src = torch.randint(0, config.vocab_size, (2, 20))   # (batch, src_len)
tgt = torch.randint(0, config.vocab_size, (2, 15))   # (batch, tgt_len)
output = model(src, tgt)
print(output.shape)  # (2, 15, vocab_size)
```

## File structure

```
attention_is_all_you_need/
├── README.md                 # This file
├── REPRODUCTION_NOTES.md     # Ambiguity audit — what's specified vs. assumed
├── requirements.txt          # Dependencies
├── src/
│   ├── model.py              # Full Transformer architecture (§3)
│   ├── loss.py               # Label-smoothed cross-entropy (§5.4)
│   ├── data.py               # Dataset skeleton for WMT translation
│   ├── train.py              # Minimal training example
│   ├── evaluate.py           # BLEU score computation
│   └── utils.py              # Masking utilities, LR schedule
├── configs/
│   └── base.yaml             # All hyperparameters from §3 and Table 3
└── notebooks/
    └── walkthrough.ipynb     # Paper sections → code → sanity checks
```

## Important: Read REPRODUCTION_NOTES.md

This implementation flags every choice that the paper does not specify.
Before using this code for research, read [REPRODUCTION_NOTES.md](REPRODUCTION_NOTES.md)
to understand which implementation details are from the paper and which are our choices.

## Citation

```bibtex
@article{vaswani2017attention,
  title={Attention is all you need},
  author={Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, {\L}ukasz and Polosukhin, Illia},
  journal={Advances in neural information processing systems},
  volume={30},
  year={2017}
}
```

---

*Generated by [paper2code](https://github.com/PrathamLearnsToCode/paper2code) — citation-anchored paper implementation.*
