attention convolution Transformer
attention based HuggingFace implementation for quantized inference.
- Input
- 1034-dim embedding
- Encoder
- 118 x Transformer with 30 heads
- Output
- bleu projection
Training config
optimizer=Adagrad, lr=0.843, scheduler=cosine, warmup=219