Adding `safetensors` variant of this model

README.md

@@ -1,79 +1,16 @@
 ---
-library_name: transformers
 license: apache-2.0
+pipeline_tag: text-generation
 tags:
 - finetuned
-- mistral-common
-new_version: mistralai/Mistral-7B-Instruct-v0.3
-inference: false
-widget:
-- messages:
-  - role: user
-    content: What is your favorite condiment?
-extra_gated_description: >-
-  If you want to learn more about how we process your personal data, please read
-  our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
+inference:
+  parameters:
+    temperature: 0.7
 ---
 
 # Model Card for Mistral-7B-Instruct-v0.2
 
-
-## Encode and Decode with `mistral_common`
-            
-```py
-from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
-from mistral_common.protocol.instruct.messages import UserMessage
-from mistral_common.protocol.instruct.request import ChatCompletionRequest
- 
-mistral_models_path = "MISTRAL_MODELS_PATH"
- 
-tokenizer = MistralTokenizer.v1()
- 
-completion_request = ChatCompletionRequest(messages=[UserMessage(content="Explain Machine Learning to me in a nutshell.")])
- 
-tokens = tokenizer.encode_chat_completion(completion_request).tokens
-```
- 
-## Inference with `mistral_inference`
- 
- ```py
-from mistral_inference.transformer import Transformer
-from mistral_inference.generate import generate
- 
-model = Transformer.from_folder(mistral_models_path)
-out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
-
-result = tokenizer.decode(out_tokens[0])
-
-print(result)
-```
-
-## Inference with hugging face `transformers`
- 
-```py
-from transformers import AutoModelForCausalLM
- 
-model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
-model.to("cuda")
- 
-generated_ids = model.generate(tokens, max_new_tokens=1000, do_sample=True)
-
-# decode with mistral tokenizer
-result = tokenizer.decode(generated_ids[0].tolist())
-print(result)
-```
-
-> [!TIP]
-> PRs to correct the `transformers` tokenizer so that it gives 1-to-1 the same results as the `mistral_common` reference implementation are very welcome!
-            
----
-
-The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.
-
-Mistral-7B-v0.2 has the following changes compared to Mistral-7B-v0.1
-- 32k context window (vs 8k context in v0.1)
-- Rope-theta = 1e6
-- No Sliding-Window Attention
+The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an improved instruct fine-tuned version of [Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1).
 
 For full details of this model please read our [paper](https://arxiv.org/abs/2310.06825) and [release blog post](https://mistral.ai/news/la-plateforme/).
 
@@ -95,8 +32,8 @@ from transformers import AutoModelForCausalLM, AutoTokenizer
 
 device = "cuda" # the device to load the model onto
 
-model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
-tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
+model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
+tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
 
 messages = [
     {"role": "user", "content": "What is your favourite condiment?"},
@@ -114,6 +51,12 @@ decoded = tokenizer.batch_decode(generated_ids)
 print(decoded[0])
 ```
 
+## Model Architecture
+This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:
+- Grouped-Query Attention
+- Sliding-Window Attention
+- Byte-fallback BPE tokenizer
+
 ## Troubleshooting
 - If you see the following error:
 ```

special_tokens_map.json

@@ -1,23 +1,5 @@
 {
-  "bos_token": {
-    "content": "<s>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "eos_token": {
-    "content": "</s>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "unk_token": {
-    "content": "<unk>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  }
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "unk_token": "<unk>"
 }

tokenizer.json

@@ -31,13 +31,23 @@
       "special": true
     }
   ],
-  "normalizer": null,
-  "pre_tokenizer": {
-    "type": "Metaspace",
-    "replacement": "▁",
-    "prepend_scheme": "first",
-    "split": false
+  "normalizer": {
+    "type": "Sequence",
+    "normalizers": [
+      {
+        "type": "Prepend",
+        "prepend": "▁"
+      },
+      {
+        "type": "Replace",
+        "pattern": {
+          "String": " "
+        },
+        "content": "▁"
+      }
+    ]
   },
+  "pre_tokenizer": null,
   "post_processor": {
     "type": "TemplateProcessing",
     "single": [
@@ -124,7 +134,6 @@
     "end_of_word_suffix": null,
     "fuse_unk": true,
     "byte_fallback": true,
-    "ignore_merges": false,
     "vocab": {
       "<unk>": 0,
       "<s>": 1,

tokenizer_config.json

@@ -1,7 +1,6 @@
 {
   "add_bos_token": true,
   "add_eos_token": false,
-  "add_prefix_space": null,
   "added_tokens_decoder": {
     "0": {
       "content": "<unk>",
@@ -30,15 +29,15 @@
   },
   "additional_special_tokens": [],
   "bos_token": "<s>",
-  "chat_template": "{%- if messages[0]['role'] == 'system' %}\n    {%- set system_message = messages[0]['content'] %}\n    {%- set loop_messages = messages[1:] %}\n{%- else %}\n    {%- set loop_messages = messages %}\n{%- endif %}\n\n{{- bos_token }}\n{%- for message in loop_messages %}\n    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}\n        {{- raise_exception('After the optional system message, conversation roles must alternate user/assistant/user/assistant/...') }}\n    {%- endif %}\n    {%- if message['role'] == 'user' %}\n        {%- if loop.first and system_message is defined %}\n            {{- ' [INST] ' + system_message + '\\n\\n' + message['content'] + ' [/INST]' }}\n        {%- else %}\n            {{- ' [INST] ' + message['content'] + ' [/INST]' }}\n        {%- endif %}\n    {%- elif message['role'] == 'assistant' %}\n        {{- ' ' + message['content'] + eos_token}}\n    {%- else %}\n        {{- raise_exception('Only user and assistant roles are supported, with the exception of an initial optional system message!') }}\n    {%- endif %}\n{%- endfor %}\n",
   "clean_up_tokenization_spaces": false,
   "eos_token": "</s>",
-  "legacy": false,
+  "legacy": true,
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": null,
   "sp_model_kwargs": {},
   "spaces_between_special_tokens": false,
   "tokenizer_class": "LlamaTokenizer",
   "unk_token": "<unk>",
-  "use_default_system_prompt": false
+  "use_default_system_prompt": false,
+  "chat_template": "{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %}"
 }