Skip to content
Snippets Groups Projects
Commit 2a14d364 authored by Victor Zimmermann's avatar Victor Zimmermann
Browse files

Second try, better results (On smaller test set.)

parent 6d229cf3
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 283 additions and 0 deletions
results/try2.log 0 → 100644
+ 283
− 0
View file @ 2a14d364
[ INFO ] Configuration - Loading /eval.properties FROM /home/students/zimmermann/Courses/ws17/fsem/absinth/WSI-Evaluator/config/eval.properties
[ INFO ] WSIEvaluator - Run started at 16:53
[ INFO ] Dataset - Loading the datasets/trial
[ INFO ] Dataset - Loading the topics data datasets/trial
[ INFO ] Dataset - Loading the subtopics data
[ INFO ] Dataset - Loading the snippets data
[ INFO ] Dataset - Loading the relations data
[ INFO ] Dataset - Loading the relations data
[ INFO ] WSIEvaluator -
===================================================
Starting the evaluation
===================================================
[ INFO ] WSIEvaluator -
===================================================
subtopic-recall@K default: 100
===================================================
[ WARN ] WSIEvaluator -
============= Query 1 : "soul food" ==============
[ INFO ] WSIEvaluator -
============== Cluster creation phase ==================
[ WARN ] WSIEvaluator - ClusteredSnippets:
================== 4 snippet clusters: ====================
The cluster 1 contains the snippets: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 17, 19, 20, 22, 23, 24, 25, 26, 27, 30, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 52, 53, 55, 56, 58, 60, 61, 63, 64, 65, 66, 67, 69, 70, 71, 72, 74, 75, 76, 78, 79, 80, 82, 83, 84, 85, 87, 88, 90, 91, 94, 95, 96, 97, 98]
The cluster 2 contains the snippets: [18, 21, 28, 36, 44, 51, 68, 73, 77, 81, 86, 92, 100]
The cluster 3 contains the snippets: [15, 31, 32, 54, 62, 99]
The cluster 4 contains the snippets: [93]
[ INFO ] WSIEvaluator -
================== Starting Evaluation ==================
Resulting list: [1, 18, 15, 93, 2, 21, 31, 3, 28, 32, 4, 36, 54, 5, 44, 62, 6, 51, 99, 7, 68, 8, 73, 9, 77, 10, 81, 11, 86, 12, 92, 13, 100, 14, 17, 19, 20, 22, 23, 24, 25, 26, 27, 30, 33, 34, 35, 37, 38, 39, 40, 41, 42, 43, 45, 46, 47, 48, 49, 50, 52, 53, 55, 56, 58, 60, 61, 63, 64, 65, 66, 67, 69, 70, 71, 72, 74, 75, 76, 78, 79, 80, 82, 83, 84, 85, 87, 88, 90, 91, 94, 95, 96, 97, 98, 16, 29, 57, 59, 89]
[ WARN ] WSIEvaluator -
================ Results of Rand Index for the topic "soul food" ===============
============ Value of Rand Index = 0.5931313131313132 =================================
============ Partial average Rand Index = 0.5931313131313132 ======
============ Value of Adjusted Rand Index = 0.14137469578255443 =================================
============ Partial average of Adjusted Rand Index = 0.14137469578255443 =================================
============ Value of Jaccard Index = 0.49396984924623116 =================================
============ Partial average of Jaccard Index = 0.49396984924623116 ======
============ Precision = 0.8, Recall = 0.8172043010752689, F1 = 0.8085106382978724 ============
[ WARN ] WSIEvaluator -
================ Statistics: ====================================
============ number of created clusters: 4 ============
============ average size of the created clusters: 23.75 ============
===========================================================================
[ WARN ] WSIEvaluator -
============= Query 2 : "the block" ==============
[ INFO ] WSIEvaluator -
============== Cluster creation phase ==================
[ WARN ] WSIEvaluator - ClusteredSnippets:
================== 4 snippet clusters: ====================
The cluster 1 contains the snippets: [2, 3, 4, 5, 6, 8, 9, 11, 13, 19, 20, 22, 23, 24, 25, 27, 28, 30, 31, 34, 35, 38, 40, 41, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 59, 61, 63, 64, 65, 67, 68, 69, 70, 73, 74, 77, 80, 81, 84, 85, 86, 89, 90, 92, 94, 96, 99]
The cluster 4 contains the snippets: [12, 21, 29, 32, 36, 37, 43, 53, 55, 57, 62, 66, 72, 75, 76, 78, 79, 82, 83, 87, 91, 93, 97, 100]
The cluster 2 contains the snippets: [7, 33, 39, 56, 71, 88, 98]
The cluster 3 contains the snippets: [14, 16, 26, 58, 60, 95]
[ INFO ] WSIEvaluator -
================== Starting Evaluation ==================
Resulting list: [2, 7, 14, 12, 3, 33, 16, 21, 4, 39, 26, 29, 5, 56, 58, 32, 6, 71, 60, 36, 8, 88, 95, 37, 9, 98, 43, 11, 53, 13, 55, 19, 57, 20, 62, 22, 66, 23, 72, 24, 75, 25, 76, 27, 78, 28, 79, 30, 82, 31, 83, 34, 87, 35, 91, 38, 93, 40, 97, 41, 100, 42, 44, 45, 46, 47, 48, 49, 50, 51, 52, 54, 59, 61, 63, 64, 65, 67, 68, 69, 70, 73, 74, 77, 80, 81, 84, 85, 86, 89, 90, 92, 94, 96, 99, 1, 10, 15, 17, 18]
[ WARN ] WSIEvaluator -
================ Results of Rand Index for the topic "the block" ===============
============ Value of Rand Index = 0.546060606060606 =================================
============ Partial average Rand Index = 0.5695959595959597 ======
============ Value of Adjusted Rand Index = 0.1164260322125791 =================================
============ Partial average of Adjusted Rand Index = 0.12890036399756677 =================================
============ Value of Jaccard Index = 0.36110321296559567 =================================
============ Partial average of Jaccard Index = 0.4275365311059134 ======
============ Precision = 0.21052631578947367, Recall = 0.7407407407407407, F1 = 0.32786885245901637 ============
[ WARN ] WSIEvaluator -
================ Statistics: ====================================
============ number of created clusters: 4 ============
============ average size of the created clusters: 23.75 ============
===========================================================================
[ WARN ] WSIEvaluator -
============= Query 3 : "stephen king" ==============
[ INFO ] WSIEvaluator -
============== Cluster creation phase ==================
[ WARN ] WSIEvaluator - ClusteredSnippets:
================== 4 snippet clusters: ====================
The cluster 1 contains the snippets: [2, 3, 6, 9, 10, 13, 14, 17, 18, 19, 20, 21, 23, 24, 26, 28, 29, 38, 39, 40, 41, 42, 44, 46, 49, 50, 51, 55, 57, 58, 59, 60, 61, 64, 65, 66, 67, 69, 70, 72, 73, 77, 78, 80, 81, 82, 83, 85, 93, 94, 97, 100]
The cluster 2 contains the snippets: [1, 4, 5, 8, 22, 27, 30, 31, 35, 37, 45, 47, 63, 68, 74, 75, 76, 79, 84, 86, 88, 89, 91, 98]
The cluster 3 contains the snippets: [7, 11, 12, 15, 16, 32, 33, 34, 36, 48, 53, 54, 56, 62, 71, 87, 95, 96, 99]
The cluster 4 contains the snippets: [25, 43, 90, 92]
[ INFO ] WSIEvaluator -
================== Starting Evaluation ==================
Resulting list: [2, 1, 7, 25, 3, 4, 11, 43, 6, 5, 12, 90, 9, 8, 15, 92, 10, 22, 16, 13, 27, 32, 14, 30, 33, 17, 31, 34, 18, 35, 36, 19, 37, 48, 20, 45, 53, 21, 47, 54, 23, 63, 56, 24, 68, 62, 26, 74, 71, 28, 75, 87, 29, 76, 95, 38, 79, 96, 39, 84, 99, 40, 86, 41, 88, 42, 89, 44, 91, 46, 98, 49, 50, 51, 55, 57, 58, 59, 60, 61, 64, 65, 66, 67, 69, 70, 72, 73, 77, 78, 80, 81, 82, 83, 85, 93, 94, 97, 100, 52]
[ WARN ] WSIEvaluator -
================ Results of Rand Index for the topic "stephen king" ===============
============ Value of Rand Index = 0.35878787878787877 =================================
============ Partial average Rand Index = 0.4993265993265994 ======
============ Value of Adjusted Rand Index = -0.009811781773756394 =================================
============ Partial average of Adjusted Rand Index = 0.08266298207379238 =================================
============ Value of Jaccard Index = 0.3525091799265606 =================================
============ Partial average of Jaccard Index = 0.4025274140461292 ======
============ Precision = 0.98989898989899, Recall = 0.98989898989899, F1 = 0.98989898989899 ============
[ WARN ] WSIEvaluator -
================ Statistics: ====================================
============ number of created clusters: 4 ============
============ average size of the created clusters: 24.75 ============
===========================================================================
[ WARN ] WSIEvaluator -
============= Query 4 : "cool water" ==============
[ INFO ] WSIEvaluator -
============== Cluster creation phase ==================
[ WARN ] WSIEvaluator - ClusteredSnippets:
================== 2 snippet clusters: ====================
The cluster 1 contains the snippets: [1, 2, 6, 7, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 28, 29, 30, 32, 33, 34, 36, 37, 38, 39, 41, 42, 43, 44, 45, 47, 48, 49, 52, 53, 54, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 83, 84, 85, 86, 88, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100]
The cluster 2 contains the snippets: [4]
[ INFO ] WSIEvaluator -
================== Starting Evaluation ==================
Resulting list: [1, 4, 2, 6, 7, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 28, 29, 30, 32, 33, 34, 36, 37, 38, 39, 41, 42, 43, 44, 45, 47, 48, 49, 52, 53, 54, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 70, 71, 72, 74, 75, 76, 77, 78, 79, 80, 81, 83, 84, 85, 86, 88, 90, 91, 92, 93, 94, 95, 96, 98, 99, 100, 3, 5, 8, 9, 10, 12, 25, 26, 27, 31, 35, 40, 46, 50, 51, 55, 56, 69, 73, 82, 87, 89, 97]
[ WARN ] WSIEvaluator -
================ Results of Rand Index for the topic "cool water" ===============
============ Value of Rand Index = 0.6270707070707071 =================================
============ Partial average Rand Index = 0.5312626262626263 ======
============ Value of Adjusted Rand Index = 0.21282220172709276 =================================
============ Partial average of Adjusted Rand Index = 0.11520278698711747 =================================
============ Value of Jaccard Index = 0.5347782258064516 =================================
============ Partial average of Jaccard Index = 0.43559011698620975 ======
============ Precision = 0.16883116883116883, Recall = 0.52, F1 = 0.25490196078431376 ============
[ WARN ] WSIEvaluator -
================ Statistics: ====================================
============ number of created clusters: 2 ============
============ average size of the created clusters: 38.5 ============
===========================================================================
[ INFO ] WSIEvaluator -
=========== Final average value of S-recall@K: =============
1 0.3958
2 0.5417
3 0.6250
4 0.6250
5 0.6250
6 0.6250
7 0.6250
8 0.6875
9 0.6875
10 0.6875
11 0.6875
12 0.6875
13 0.7708
14 0.8542
15 0.8542
16 0.8542
17 0.8542
18 0.8542
19 0.8542
20 0.8542
21 0.8542
22 0.8542
23 0.8542
24 0.9167
25 0.9167
26 0.9167
27 0.9167
28 0.9167
29 0.9167
30 0.9167
31 0.9167
32 0.9167
33 0.9167
34 0.9167
35 0.9167
36 0.9167
37 0.9167
38 0.9167
39 0.9167
40 0.9167
41 0.9167
42 0.9167
43 0.9167
44 0.9167
45 0.9167
46 0.9167
47 0.9167
48 0.9167
49 0.9167
50 0.9167
51 0.9167
52 0.9167
53 0.9167
54 0.9167
55 0.9167
56 0.9167
57 0.9167
58 0.9167
59 0.9167
60 0.9167
61 0.9167
62 0.9167
63 0.9167
64 0.9167
65 0.9167
66 0.9167
67 0.9167
68 0.9167
69 0.9167
70 0.9167
71 0.9167
72 0.9167
73 0.9167
74 0.9167
75 0.9167
76 0.9167
77 0.9167
78 0.9167
79 0.9167
80 1.0000
81 1.0000
82 1.0000
83 1.0000
84 1.0000
85 1.0000
86 1.0000
87 1.0000
88 1.0000
89 1.0000
90 1.0000
91 1.0000
92 1.0000
93 1.0000
94 1.0000
95 1.0000
96 1.0000
97 1.0000
98 1.0000
99 1.0000
100 1.0000
=========== Final average value of S-precision@r: =============
0.4000 0.6068
0.4500 0.4603
0.5000 0.3583
0.5500 0.2865
0.6000 0.2489
0.6500 0.2389
0.7000 0.1527
0.7500 0.1197
0.8000 0.1126
0.8500 0.1066
0.9000 0.1013
0.9500 0.0966
1.0000 0.0267
=========== Final average value of F1: =====================
average F1 = 0.6353649811275954
=========== Final average value of Rand Index: =============
average Rand Index = 0.5312626262626263
=========== Final average value of Adjusted Rand Index: ====
average Adj Rand Index = 0.11520278698711747
=========== Final average value of Jaccard Index: ==========
average Jaccard Index = 0.43559011698620975
================ Statistics: ====================================
============ average number of created clusters: 3.5
============ average cluster size: 27.6875
[ WARN ] WSIEvaluator - Simulation started at: 16:53 and completed at 16:53
Execution time: 2.1 sec
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment