Training in progress, step 200

README.md

@@ -0,0 +1,70 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen2-0.5B
+tags:
+- generated_from_trainer
+metrics:
+- accuracy
+model-index:
+- name: llm2vec-Qwen2-0.5B-shopping-Quar-v2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# llm2vec-Qwen2-0.5B-shopping-Quar-v2
+
+This model is a fine-tuned version of [Qwen/Qwen2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 3.1151
+- Accuracy: 0.4539
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 4
+- eval_batch_size: 4
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 1
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step  | Validation Loss | Accuracy |
+|:-------------:|:------:|:-----:|:---------------:|:--------:|
+| 3.4292        | 0.0510 | 1000  | 3.3647          | 0.4207   |
+| 3.3217        | 0.1019 | 2000  | 3.2997          | 0.4283   |
+| 3.274         | 0.1529 | 3000  | 3.2483          | 0.4345   |
+| 3.2467        | 0.2038 | 4000  | 3.1933          | 0.4407   |
+| 3.2093        | 0.2548 | 5000  | 3.1931          | 0.4439   |
+| 3.1807        | 0.3057 | 6000  | 3.1661          | 0.4476   |
+| 3.1978        | 0.3567 | 7000  | 3.1433          | 0.4486   |
+| 3.161         | 0.4076 | 8000  | 3.1091          | 0.4539   |
+| 3.1554        | 0.4586 | 9000  | 3.1044          | 0.4529   |
+| 3.1405        | 0.5095 | 10000 | 3.1151          | 0.4539   |
+
+
+### Framework versions
+
+- Transformers 4.43.4
+- Pytorch 1.12.0+cu102
+- Datasets 2.20.0
+- Tokenizers 0.19.1

adapter_config.json

@@ -23,13 +23,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "v_proj",
-    "down_proj",
-    "gate_proj",
     "o_proj",
-    "q_proj",
+    "k_proj",
     "up_proj",
-    "k_proj"
+    "gate_proj",
+    "v_proj",
+    "down_proj",
+    "q_proj"
   ],
   "task_type": null,
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d7170d45d501358e51b89ba4a4ea47bbc38b562883275dd326314071eeaf60b6
+oid sha256:e1edd994befa0536dc879b728734c37101b72917e103799a9050bbffee1db3da
 size 35235088

runs/Sep27_14-08-28_HPC2/events.out.tfevents.1727435431.HPC2.125065.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e1ffe0999f614c9fb6ef3e5447bddb95b161159291631d96ff827565c864ab48
-size 11990
+oid sha256:e8e899fedc0fd0a4b2397d37be2c58174b04c6d48ff7d1cee356f0de1fcba744
+size 12878

runs/Sep28_12-24-15_HPC2/events.out.tfevents.1727515552.HPC2.609693.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e010ae3ef78f1e67c9119e9eac41af70d5221b392c0ca65dbce848dd1eb3232
+size 5074

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:758e334d4c9315a57b3b32f173ecb1b58a3fa057669bdbfafd012ce6ab78c15f
+oid sha256:bc0d08bb1abb85f267b5bf85ca5bd3c6fa64a03d8d852d595d3d9d47c55bdfe1
 size 4847