Fine-tuning is the process where a pre-trained language model is trained to perform a specialized task. In our case BERT needs to be fine-tuned to predict the relation between the two components of a nominal compound based on the sentence they occur in.
...
...
@@ -81,6 +81,69 @@ fineBert.py t file_test.csv
### Results
The resulting model of the fine-tuning process is stored in a directory named after the input file, batch size and learning rate. We output numpy arrays with true labels and predictions which are saved in the working directory. For better clarity we moved them in the results directory.
## Baselines
For all 17 data variations, the fine-tuned model was compared to each of the three baselines. As for the probing, the baselines used were a majority baseline, a random uniform prediction, and a random classification weighted by category frequency. The model's performance was compared to each baseline using McNemar's test for nominal data. As depicted in the table below, the performance of the model was slightly better for the coarse-grained than for fine-grained relations. The prediction of the model was significantly better than the uniform baseline for both granularity levels and than the majority baseline for coarse-grained relations for all data set variations. That the predictions is not better than the stratified random baseline for some data variations, further illustrates that the model is not simply memorizes nouns, but indeed seems to \emph{understand} the relation between the noun compound elements.
<br>
The tables to be expended show the p-values of McNemar's test for all combinations of data set variation and baselines, where the model was significantly better. Sparces indicate that where was no significant difference in prediction or the baseline was significantly better.
#### Fine-Grained Relations
<details>
<summarymarkdown="span"><i>🔎Expend to see results from McNemar's all on the fine-grained relations for all data variations.</i></summary>
The fine-tuning process is performed using the BertForSequenceClassification transformer which provides a special layer in its neural network for classification issues. For practical purposes and to improve performance we make use of the following transformer and pytorch classes - `DataLoader`, `torch.optim.AdamW` and `get_scheduler`:
-`DataLoader` provides an iterable over the data with no need for a for-loop
...
...
@@ -152,60 +215,3 @@ The last testing experiment was on the test set with the components flipped. Int
</div></p>
## Evaluation
### Fine-Grained Relations
<details>
<summarymarkdown="span"><i>🔎Expend to see results from McNemar's test for all fine-grained model.</i></summary>
