Loading check_fasttext_vocab.py 0 → 100644 +17 −0 Original line number Diff line number Diff line """Find out vocab counts and positions for names in names lists, in German fasttext model.""" import fasttext import pandas as pd df = pd.read_csv("../data/names_nationality_data.csv") df["in_fasttext_vocab"] = 0 model = fasttext.load_model("cc.de.300.bin") for index, row in df.iterrows(): name = row['name'] if name in model.words: df.at[index, "in_fasttext_vocab"] = 1 df.to_csv("../data/names_nationality_fasttext.csv", index=False) count_name_occurrences.py→count_name_occurrences_wikipedia.py +0 −0 File moved. View file Loading
check_fasttext_vocab.py 0 → 100644 +17 −0 Original line number Diff line number Diff line """Find out vocab counts and positions for names in names lists, in German fasttext model.""" import fasttext import pandas as pd df = pd.read_csv("../data/names_nationality_data.csv") df["in_fasttext_vocab"] = 0 model = fasttext.load_model("cc.de.300.bin") for index, row in df.iterrows(): name = row['name'] if name in model.words: df.at[index, "in_fasttext_vocab"] = 1 df.to_csv("../data/names_nationality_fasttext.csv", index=False)