@@ -91,7 +91,9 @@ As mentioned before, all evalution metrics confirmed our initial study conjectur
During the annotation phase of our gold data (and especially during the calculation of the Inter-Annotator-Agreement), we had already noticed that assigning the proper and most fitting class for Maslow and especially for Reiss was not as straightforwards as expected. The coarse-grained nature of Maslow categories provided us much less options to choose from, which in turn made our annotation itself more confident. Reiss motives, however, were much more difficult approach as they were much more fine-grained and provided such a wide range of possible options that, at times, it became very difficult us to choose the right motive. In addition to that, two or even more motives would have been equally fitting for one essay which increased the difficulty of the task. This notion was first supported by our Inter-Annotator-Agreement values, in which the annotations for Maslow categories showed much higher agreement between us than for those of the Reiss motives. Unsurprisingly, the system matched our pattern and intution and scored significantly higher for the coarse-grained Maslow categories than for the fine-grained Reiss motives. The conclusion which we can draw from this is that, given two groups of classes to choose from, the system will perform better on the coarse-grained classes and show be less effective on fine-grained classes. For future research, it could be of special interest to shift the focus to the correct classification of fine-grained classes and categories, as fulfillment of this task could also potentially entail the correct classification of the coarse-grained classes.
2. Difficulty of texts
Since this last paragraph contains all the necessary information about the text, we consider only that one in our project. This, of course, makes the task more difficult to approach as it excludes the context prior to the last paragraph and thus disregards any reference occurring in the last paragraphs to prior arguments. Especially for texts of argumentative nature, this can negatively impact the system performance in a significant way, as prior context can be of relevance in some cases.
