Skip to content
Snippets Groups Projects
Commit f034c906 authored by pracht's avatar pracht
Browse files

HF Token instruction

parent b5897466
No related branches found
No related tags found
No related merge requests found
......@@ -4,19 +4,24 @@ Extracted Literal Search is a simple method for Retrieval Augmented Generation (
For small datasets where most of the data fields are known literals, this method is easier to implement and might provide better results than Similarity Search of embeddings.
## Installation
Make sure you have `poetry` installed.
> pip install --local poetry
Install the package.
> poetry install
Export your Huggingface token.
> export HUGGING_FACE_HUB_TOKEN=<TOKEN>
Run the vLLM server. For example with a quantized Llama-2 model.
> poetry run python -m outlines.serve.serve --model="TheBloke/Llama-2-7b-Chat-GPTQ" -q gptq
> poetry run python -m outlines.serve.serve --model="TheBloke/Llama-2-7b-Chat-GPTQ" -q gptq
## Usage
Define your domain specific prompts.
\ No newline at end of file
Define your domain specific prompts.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment