Update project_report.md

## Results and Evaluation

#### Annotations and Maslow hierarchy of needs

Our first step in this project, as mentioned earlier, was to manually annotate all essays from the Student essay corpus [Student essay Corpus](https://www.informatik.tu-darmstadt.de/ukp/research_6/data/argumentation_mining_1/argument_annotated_essays_version_2/index.en.jsp). Here we noticed that the Maslow categories exclusively represent human needs. However, we also found some essays that were not about humans, but about the needs of other living or non-living beings. For our project we decided to extend Maslow's motives to other, non-human existences. <br> In principle however we would find a revised, more existence-needs integrating pyramid meaningful. Above all in today's time, where animal protection, nature protection etc. are large important topics, which also is much written and reported about, it is important that these needs are filtered out and included. <br> **The only question left when revising the Maslow pyramind is: would one like to regard e.g. the animal protection as human need - thus as need of humans to protect the environment or as the need of the animal to live?** 

Our first step in this project, as mentioned earlier, was to manually annotate all essays from the [student essay corpus](https://www.informatik.tu-darmstadt.de/ukp/research_6/data/argumentation_mining_1/argument_annotated_essays_version_2/index.en.jsp). We noticed that the Maslow categories exclusively represent human needs. However, we also found some essays that were not about humans, but about the needs of other living or non-living beings. For our project we decided to extend Maslow's motives to other, non-human existences. <br> In principle however we would find a revised, more existence-needs integrating pyramid meaningful. Above all in today's time, where animal protection, nature protection etc. are important topics that generate a lot of content and text, it is important that these needs are filtered out and included. <br> **The only question left when revising the Maslow pyramind is: would one like to regard e.g. the animal protection as human need - thus as need of humans to protect the environment or as the need of the animal to live?** 

<br>

Another observation we made during our annotation was that the Maslow and Reiss categories are not so simple and not clearly assignable. Already during the annotation we all noticed that often more than one category is applicable. While we decided on the basis of our gut feeling what was most applicable for us, we realized already when comparing with the others that in some examples we had even assigned four different needs. And each one fit in its own way. Then, when we used Fleiss Kappa to calculate our inter-annotator agreement, we found that we came up with an agreement of about 0.7 for Maslow and 0.4 for Reiss. This result made it clear that even if we interpreted the human needs behind an essay in the same way, it would be difficult to agree on the Reiss motives, which are even more concrete than the Maslow motives. Which motives one suspects behind a text written by someone else depends too much on the person reading it and his personal feelings.

Another observation we made during our annotation was that the Maslow and Reiss categories are not so simple and not clearly assignable. During the annotation we all noticed that often more than one category is applicable. While we decided on the basis of our gut feeling what was most applicable for us, we realized when comparing with the others that in some examples we had sometimes assigned four different needs. And each one fit in its own way. Then, when we used Fleiss Kappa to calculate our inter-annotator agreement, we found that we ended up with an agreement of about 0.7 for Maslow and 0.4 for Reiss. This result made it clear that even if we interpreted the human needs behind an essay in the same way, it would be difficult to agree on the Reiss motives, which are even more fine grain than the Maslow motives. Which motives one suspects behind a text written by someone else depends largely on the person reading it and their personal feelings.

<br>

After extracting the knowledge paths, we found that ConceptNet also has a problem with assigning Reiss motives. While the Maslow motives were often well assigned, ConceptNet showed a clear preference for some motives of Reiss. For example, the Reiss motive social was assigned much more often and also in places where a person would have already tended towards love/belonging. Thus ConceptNet has difficulties distinguishing social from concrete feelings such as love.

After extracting the knowledge paths, we found that ConceptNet also has a problem with assigning Reiss motives. While the Maslow motives were often well assigned, ConceptNet showed a clear preference for some motives of Reiss. For example, the Reiss motive "social" was assigned much more often and also in places where a person would have tended towards love/belonging. Thus ConceptNet has difficulties distinguishing social from concrete feelings such as love.

#### Assigning the human needs

After extracting the knowledgepaths, we found that the neural model of @DebjitPaul was not immediately executable without errors. Again there were a lot of version problems, the versions given in the [github](https://github.com/debjitpaul/Multi-Hop-Knowledge-Paths-Human-Needs) are outdated and every version change caused a problem with another package (see [Problems that accured](#problems-that-accured)). Since we didn't want to invest all our time again in finding out the correct versions of another project and @debjitpaul couldn't help with the faulty version specifications either, we decided to develop our own method for this. Since the knowledgepaths are already returned according to their expressiveness, we applied them in *humans_needs_assginer.py* by using and assigning the human needs of the first path we found.