Merge branch 'master' of...

Merge branch 'master' of https://gitlab.cl.uni-heidelberg.de/friebolin/swp-data-augmentation-for-metonymy-resolution

Code/submit_template.sh

+#!/bin/bash
+#
+#SBATCH --job-name=JOBNAME
+#SBATCH --output=output_file.txt
+#SBATCH --mail-user=mail@cl.uni-heidelberg.de
+#SBATCH --mail-type=ALL
+#SBATCH --partition=PARTITION
+#SBATCH --mem 9G
+#SBATCH --gres=gpu:1
+#SBATCH --ntasks=1
+
+#JOB STEPS
+#srun hostname
+
+# source desired venv here:
+source venv/bin/activate
+
+cd /path/to/code/repository/swp-data-augmentation-for-metonymy-resolution/Code
+
+python main.py --architecture MODEL_TO_TRAIN --model_type CLASSIFICATION_MODEL -t PATH/TO/TRAIN_SET.txt -v PATH/TO/TEST_SET.txt --tokenizer TOKENIZER -max MAXIMUM_SEQUENZE_LENGHT -rs RANDOM_SEED -lr LEARNING_RATE -e EPOCHS --train_loop TRAINER  -b TRAIN_BATCH_SIZE -tb TEST_BATCH_SIZE -sd PATH/TO/RESULT/file.txt
+
+#for methods MixUp and TMix add arguments as shown in main.py
\ No newline at end of file

README.md

@@ -172,57 +172,71 @@ pip install -r requirements.txt
 ***
 ## ⚙️ Usage <a name="usage"></a>
 
-🚀 Launch our application by following the steps below:
-
-[welche argumente genau?]
+#### 🚀 Launch our application by following the steps below:
 
 ```bash
 ./main.py <COMMAND> <ARGUMENTS>...
 ```
 
-For `<COMMAND>` you must enter one of the commands you find in the list below, where you can also find an overview about necessary `<ARGUMENTS>`.
+For `<COMMAND>` you must enter one of the commands you find in the list below, where you can also find an overview about possible `<ARGUMENTS>`. 
+
+ℹ️ The icons indicate if a command is mandatory (🔛). 🍸 indicates that this command is mandatory for *MixUp*, 🌐 describes mandatory commands for *TMix*.
 
 | Command | Functionality | Arguments |
 | ------- | ------------- |-----------|
-|**General**|
-|**`--architecture`** | Defines which model is used. | Choose `bert-base-uncased` or `roberta`  |
-|**`--model_type`** | How to initialize the Classification Model | Choose `separate` or `one`  |
-|**`--mixlayer`**| Specify in which `layer` the interpolation takes place. Only select one layer at a time. | Choose from ${0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}$ |
-|**`--tokenizer`**|Which tokenizer to use when preprocessing the datasets.|Choose `swp` for our tokenizer, `li ` for the tokenizer of Li et al. [^6], or `salami` for the tokenizer used by another [student project](https://gitlab.cl.uni-heidelberg.de/salami-hd/salami/-/tree/master/)|
-|**`-max`**/**`--max_length`**|Defines the maximum sequence length when tokenizing the sentences.|⚠️ Always choose 256 for *TMix* and 512 for the other models.|
-|**`--train_loop`**|Defines which train loop to use.|Choose `swp` for our train loop implementation and `salami` for the one of the [salami](https://gitlab.cl.uni-heidelberg.de/salami-hd/salami) student project.|
-|**`-e`**/**`--epochs`**|Number of epochs for training.||
-|**`-lr`**/**`--learning_rate`**|Learning rate for training.|`type=float`|
-|**`-rs`**/**`--random_seed`**|Random seed for initialization of the model.|Default is $42$.|
+| <center> **General** </center>|
+|🔛 **`--architecture`** | Defines which model is used. | Choose `bert-base-uncased` or `roberta`  |
+|🔛 **`--model_type`** | How to initialize the Classification Model | Choose `separate` or `one`  |
+|🔛 **`--tokenizer`**|Which tokenizer to use when preprocessing the datasets.|Choose `swp` for our tokenizer, `li ` for the tokenizer of Li et al. [^6], or `salami` for the tokenizer used by another [student project](https://gitlab.cl.uni-heidelberg.de/salami-hd/salami/-/tree/master/)|
+|**`-tc`**/**`--tcontext`**|Whether or not to preprocess the training set with context.||
+|**`-vc`**/**`--vcontext`**|Whether or not to preprocess the test set with context.||
+|🔛 **`-max`**/**`--max_length`**|Defines the maximum sequence length when tokenizing the sentences.|Typically choose $256$ or $512$.|
+|🔛 **`--train_loop`**|Defines which train loop to use.|Choose `swp` for our train loop implementation and `salami` for the one of the [salami](https://gitlab.cl.uni-heidelberg.de/salami-hd/salami) student project.|
+|🔛 **`-e`**/**`--epochs`**|Number of epochs for training.||
+|🔛 **`-lr`**/**`--learning_rate`**|Learning rate for training.|`type=float`|
+|**`-lrtwo`**/**`--second_learning_rate`**| Separate learning rate for multi layer perceptron.|Default is `None`.|
+|**`--mlp`**| Whether or not to use two layer MLP as classifier.| |
+|🔛 **`-rs`**/**`--random_seed`**|Random seed for initialization of the model.|Default is $42$.|
 |**`-sd`**/**`--save_directory`**|This option specifies the destination directory for the output results of the run.||
 |**`-msp`**/**`--model_save_path`**|This option specifies the destination directory for saving the model.|We recommend saving models in [Code/saved_models](Code/saved_models).|
-|**`-tc`**/**`--tcontext`**|Whether or not to preprocess the training set with context.||
 |**`--masking`**|Whether or not to mask the target word.||
-|**`-lambda`**/**`--lambda_value`**|Speficies the lambda value for interpolation of *MixUp* and *TMix*|Default is $0.4$, `type=float`|
-|**MixUp** specific|
-|**`-mixup`**/**`--mix_up`**| Whether or not to use *MixUp*. If yes, please specify `lambda` and `-mixepoch`| |
-|**`-mixepoch`**/**`--mixepoch`**|Specifies the epoch(s) in which to apply *MixUp*.|Default is `None`|
-|**TMix** specific|
-|**`--tmix`**| Whether or not to use *TMix*. If yes, please specify `layer` and `-lambda`| |
-|**Datasets** specific|
-|**`-t`**/**`"--train_dataset`**|Defines which dataset is chosen for training.|Choose any of the datasets from [original_datasets](data/original_datasets), [fused_datasets](data/fused_datasets) or [paraphrases](data/paraphrases)|
-|**`-v`**/**`--test_dataset`**|Defines which dataset is chosen for testing.|Choose from ["semeval_test.txt"](data/original_datasets/semeval_test.txt), ["companies_test.txt"](data/original_datasets/companies_test.txt) or ["relocar_test.txt"](data/original_datasets/relocar_test.txt)|
+|🌐 **`--mixlayer`**| Specify in which `layer` the interpolation takes place. Only select one layer at a time. | Choose from ${0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11}$ |
+|🍸, 🌐 **`-lambda`**/**`--lambda_value`**|Speficies the lambda value for interpolation of *MixUp* and *TMix*|Choose any value between $0$ and $1$, `type=float`|
+| <center>| **MixUp** specific </center>||
+|🍸 **`-mixup`**/**`--mix_up`**| Whether or not to use *MixUp*. If yes, please specify `lambda` and `-mixepoch`| |
+|🍸 **`-mixepoch`**/**`--mixepoch`**|Specifies the epoch(s) in which to apply *MixUp*.|Default is `None`|
+| <center>| **TMix** specific </center>| Default is `None`.|
+|🌐 **`--tmix`**| Whether or not to use *TMix*. If yes, please specify `-mixlayer` and `-lambda`| |
+| <center>| **Datasets** specific </center>||
+|🔛 **`-t`**/**`"--train_dataset`**|Defines which dataset is chosen for training.|Choose any of the datasets from [original_datasets](data/original_datasets), [fused_datasets](data/fused_datasets) or [paraphrases](data/paraphrases)|
+|🔛 **`-v`**/**`--test_dataset`**|Defines which dataset is chosen for testing.|Choose from ["semeval_test.txt"](data/original_datasets/semeval_test.txt), ["companies_test.txt"](data/original_datasets/companies_test.txt) or ["relocar_test.txt"](data/original_datasets/relocar_test.txt)|
 |**`--imdb`**| Whether or not to use the [IMDB](https://huggingface.co/datasets/imdb) dataset. Note that this is only relevant for validating our *TMix* implementation.||
-|**`-b`**/**`--batch_size`**|Defines the batch size for the training process.|Default is $32$.|
-|**`-tb`**/**`--test_batch_size`**|Specifies the batch size for the test process.|Default is $16$.|
+|🔛 **`-b`**/**`--batch_size`**|Defines the batch size for the training process.|Default is $32$.|
+|🔛 **`-tb`**/**`--test_batch_size`**|Specifies the batch size for the test process.|Default is $16$.|
+
 
+#### 📝 If you want to use our *backtranslation* code, you must execute the following:
 
-extra: BT and inference
+```bash
+python3 Code/backtranslate.py
+```
+
+#### 🎥 If you want to see a demo of our model, you can enter your own sentence and let the model predict if a target word is used in its `literal` or `non-literal` sense:
+
+```bash
+python3 inference.py
+```
 
+<img src="documentation/images/demo.png" width="80%" height="80%">
 
-[ADD screenshot of demo?]
 ***
 
 ## 🏯 Code-Structure <a name="code-structure"></a>
 
 - ⚙️ [`requirements.txt`](requirements.txt): All necessary modules to install.
-- 📱 [`main.py`](main.py): Our main code file which does ...
-- 💻 [`Code`](code): Here, you can find all code files for our different models and data augmentation methods.
+- 📱 [`main.py`](main.py): Our main code file is responsible for organizing input options and calling the necessary functions to preprocess datasets, train the model, and evaluate it on a test set.
+- 🎥 [`inference.py`](inference.py): Run a demo version of our to test if an input sentence contains a metonymy.
+- 💻 [`Code`](code): Here, you can find all code files for our different models and data augmentation methods, as well as a [`submit_template.sh`](Code/submit_template.sh).
 - 📀 [`data`](data): Find all datasets in this folder.
     - 🗂️ [`backtranslations`](data/backtranslations): Contains unfiltered generated paraphrases.
     - 🗂️ [`fused_datasets`](data/fused_datasets): Contains original datasets fused with filtered paraphrases. Ready to be used for training the models.