Commit ca17e89c authored by holzinger's avatar holzinger
Browse files

Update README.md

parent 7694043a
Loading
Loading
Loading
Loading
+4 −4
Original line number Diff line number Diff line
@@ -193,12 +193,12 @@ class default_trainer(object):

Zum starten des LSTMS einfach in den ordner caption_lib navigieren und von dort aus [start.py](/caption_lib/start.py) ausführen.

##### Attention
##### visualisation  
#### Attention
#### visualisation  
 ![Attention_good](/Demo/Attention_good.png)  
Here we can see that the attention reacts on specific words and if it does not react at all then we can assume that there was a learning only from the captions not from the vector we provided.

##### Bad visualisation  
#### Bad visualisation  
 ![Attention_bad](/Demo/Attention_bad.png)  
The attention can't be mapped correctly because we do not know the dimensions of the pictures and how the CNN changed the data during the learning.
We see here that "surfer" and "riding" is in the same area but we do not know how to correctly map it on the surfer in the picture.
@@ -208,7 +208,7 @@ We see here that "surfer" and "riding" is in the same area but we do not know ho
  - early information on how to use attention after a two model approach with CNN and LSTM is hard to understand
  - Mapping of attention and picture data

##### Example captionresult  
#### Example captionresult  
 ![000000532690.jpg](/Demo/000000532690.jpg)  
singlelabel:  
“A man is holding a toothbrush in his mouth with his mouth open . <END>