@@ -361,6 +361,17 @@ while *Multilabel* recognized the objects better, but had slightly more errors i
## Conclusion
We used several [pretrained image classification networks](#retraining) and retrained them onto the categories from the [coco dataset](#dataset). Here the *Multilabel* approach proved
best for capturing the content of the images (and could have turned out to be overall best if trained for longer) and the *Singlelabel* approach was more fitted onto the language model.
To gain a grasp on the learned features we utilized [lucid](#visualization-lucid) by visualizing various activation grids, but were confronted with several problems and an output
which is stunning but sometimes hard to interpret.
After [extracting the feature vectores](#extraction) for a subset of our images in our dataset (performance issues!), we used this as input for our [LSTM](#lstm) and trained it.
At this step we implemented [attention](#attention), but ran into problems because we decided earlier to extract the 2048 dimensional feature vector, which could not backtrack the
input image proportions that varied to a high degree.
This is reflected in our [evaluation](#evaluation) scores. As automatic scores are unreliable for that task we would have loved to do a manual evaluation - but that is left for
future work, as well as proper implementation of attention, beam search, parameter tuning for the LSTM, extraction of all feature vectors from the dataset and extended
training time.
## References
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015). **Show and tell: A neural image caption generator**. In *Proceedings of the IEEE conference on computer vision and pattern recognition* (pp. 3156-3164).
