Notus is a collection of fine-tuned models using Direct Preference Optimization (DPO) and related RLHF techniques following a data-first approach. This model is the first version, fine-tuned with DPO on top of alignment-handbook/zephyr-7b-sft-full, which is the SFT model produced to create HuggingFaceH4/zephyr-7b-beta.
More information at argilla/notus-7b-v1.
This repository contains the notus-7b-v1 weights in npz format suitable for use with Apple's MLX framework.
pip install mlx
pip install huggingface_hub hf_transfer
git clone https://github.com/ml-explore/mlx-examples.git
cd mlx-examples
# Download model
export HF_HUB_ENABLE_HF_TRANSFER=1
huggingface-cli download --local-dir-use-symlinks False --local-dir notus-7b-v1-mlx alvarobartt/notus-7b-v1-mlx
# Run example
python llms/mistral/mistral.py --model-path notus-7b-v1-mlx --prompt "My name is"
Please, refer to the original model card for more details on Notus 7B v1.
Please note that this model expects a specific prompt structure.
<|system|>
{system_prompt}</s>
<|user|>
{user_prompt}</s>
<|assistant|>
Here is an example:
<|system|>
You are a pirate chatbot who always responds with Arr!</s>
<|user|>
There's a llama on my lawn, how can I get rid of him?</s>
<|assistant|>