From f034c906316d2b56821dd4386d3353553dfb25c9 Mon Sep 17 00:00:00 2001 From: Christoph Pracht <pracht@cl.uni-heidelberg.de> Date: Sat, 24 Feb 2024 20:13:39 +0100 Subject: [PATCH] HF Token instruction --- elise/README.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/elise/README.md b/elise/README.md index 54f4b02..9da8853 100644 --- a/elise/README.md +++ b/elise/README.md @@ -4,19 +4,24 @@ Extracted Literal Search is a simple method for Retrieval Augmented Generation ( For small datasets where most of the data fields are known literals, this method is easier to implement and might provide better results than Similarity Search of embeddings. - ## Installation Make sure you have `poetry` installed. + > pip install --local poetry Install the package. + > poetry install +Export your Huggingface token. + +> export HUGGING_FACE_HUB_TOKEN=<TOKEN> + Run the vLLM server. For example with a quantized Llama-2 model. -> poetry run python -m outlines.serve.serve --model="TheBloke/Llama-2-7b-Chat-GPTQ" -q gptq +> poetry run python -m outlines.serve.serve --model="TheBloke/Llama-2-7b-Chat-GPTQ" -q gptq ## Usage -Define your domain specific prompts. \ No newline at end of file +Define your domain specific prompts. -- GitLab