19

config/gigaword-L10_1500_then_AMR.json

@@ -7,9 +7,9 @@
   "print_every": 1,
   "eval_every": 100,
   "save_every": 100,
-  "max_val_steps": 2000,
+  "max_val_steps": 3000,
   "max_train_seconds": null,
-  "max_train_steps": 2000,
+  "max_train_steps": 3000,
   "batch_size": 4,
   "learning_rate": 1e-05,
   "k_samples": 100,

wesenberg/results/gigaword-L10_2000_then_AMR/statistics.txt

+
+#--------------------------------------
+Model:					gigaword-L10_2000_then_AMR
+Baseline-Model:			gigaword-L10_8000
+Error:					1
+
+1. Summary/Gold -- Wie nah am Gold-Standard
+	Model:
+		Rouge 1:		0.264
+		Rouge 2:		0.094
+		Rouge L:		0.24
+		AMR:			0.374
+	Baseline:
+		Rouge 1:		0.295
+		Rouge 2:		0.095
+		Rouge L:		0.263
+		AMR:			0.383
+
+2. Summary/Source -- Macht das Model was es soll?
+	Model AMR+SMATCH:
+		F1:				0.565
+		Precision:		0.442
+		Recall:			0.857
+
+	Baseline AMR+SMATCH:
+		F1:				0.538
+		Precision:		0.433
+		Recall:			0.768
+
+3. Summary/Baseline -- Wie unterschiedlich sind die Outputs vom Model und der Baseline
+	Rouge 1:
+		mean:			0.766
+		median:			0.783
+		st.dev.:		0.138
+		variance:		0.019
+
+	AMR+SMATCH:
+		mean:			0.671
+		median:			0.679
+		st.dev.:		0.181
+		variance:		0.033
+
+	Levenshtein: (1 char edit = 1 cost)
+		mean:			25.991
+		median:			25.0
+		st.dev.:		15.075
+		variance:		227.247
+
+	Jaccard-Similarity: (Gleiches geteilt Alles)
+		mean:			0.342
+		median:			0.286
+		st.dev.:		0.255
+		variance:		0.065
+
+	Jaccard-Distance: (Ungleiches geteilt Alles)
+		mean:			0.653
+		median:			0.706
+		st.dev.:		0.256
+		variance:		0.065
+
+4. Summary -- Guete der Fluency (Bleurt)
+	Model:
+		mean:			-0.607
+		median:			-0.717
+		st.dev.:		0.421
+		variance:		0.177
+	Baseline:
+		mean:			-0.619
+		median:			-0.717
+		st.dev.:		0.363
+		variance:		0.132
\ No newline at end of file