Loading paws.py 0 → 100644 +16 −0 Changes for paws.py: 16 added lines, 0 removed lines. Original line number Diff line number Diff line from datasets import load_dataset from sentence_transformers import SentenceTransformer dataset = load_dataset('paws', 'labeled_final', split='test') dataset.set_format(type='pandas') foo = dataset[:] model = SentenceTransformer('models/stsb-bert-large/') # predict dataset instances with model and evaluate with confusion matrix # calculate spearman rank correlation between cosine-similarity of the sentence embeddings and the gold labels # (or if too hard pearson correlation) # use code from evaluation_stsbenchmark.py Loading
paws.py 0 → 100644 +16 −0 Changes for paws.py: 16 added lines, 0 removed lines. Original line number Diff line number Diff line from datasets import load_dataset from sentence_transformers import SentenceTransformer dataset = load_dataset('paws', 'labeled_final', split='test') dataset.set_format(type='pandas') foo = dataset[:] model = SentenceTransformer('models/stsb-bert-large/') # predict dataset instances with model and evaluate with confusion matrix # calculate spearman rank correlation between cosine-similarity of the sentence embeddings and the gold labels # (or if too hard pearson correlation) # use code from evaluation_stsbenchmark.py