Commit 52b6119a authored by Sergey Edunov's avatar Sergey Edunov
Browse files

spelling

parent 2c18c273
Loading
Loading
Loading
Loading
+4 −4
Changes for data/README.md: 4 added lines, 4 removed lines.
Original line number Diff line number Diff line
Sample data processing scripts for FAIR Sequence-to-Sequence Toolkit
Sample data processing scripts for the FAIR Sequence-to-Sequence Toolkit

These scripts provide an example of pre-processing data for NMT task.
These scripts provide an example of pre-processing data for the NMT task.

# prepare-iwslt14.sh

@@ -35,7 +35,7 @@ $ python generate.py data-bin/iwslt14.tokenized.de-en \

# prepare-wmt14en2de.sh

Provides an example of pre-processing for WMT'14 English to German translation task. By default it will produce a dataset that was modeled after ["Attention Is All You Need" by Vaswani et al.](https://arxiv.org/abs/1706.03762) that includes news-commentary-v12 data.
Provides an example of pre-processing for the WMT'14 English to German translation task. By default it will produce a dataset that was modeled after ["Attention Is All You Need" by Vaswani et al.](https://arxiv.org/abs/1706.03762) that includes news-commentary-v12 data.

To use only data awailable in WMT'14 or to replicate results obtained in the original paper ["Convolutional Sequence to Sequence Learning" by Gehring et al.](https://arxiv.org/abs/1705.03122) run it with --icml17 instead:

@@ -72,7 +72,7 @@ $ python generate.py data-bin/wmt14_en_de \

# prepare-wmt14en2fr.sh

Provides an example of pre-processing for WMT'14 English to French translation task.
Provides an example of pre-processing for the WMT'14 English to French translation task.

Example usage: