Training in progress, step 180, checkpoint

checkpoint-180/README.md

@@ -0,0 +1,202 @@
+---
+library_name: peft
+base_model: Weni/WeniGPT-Agents-Mistral-1.0.6-SFT-merged
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.10.0

checkpoint-180/adapter_config.json

@@ -0,0 +1,31 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Weni/WeniGPT-Agents-Mistral-1.0.6-SFT-merged",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "o_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

checkpoint-180/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71508fd17885ae4c7155549b89a7c888830bffd671b6288237dacc07799a0372
+size 27297032

checkpoint-180/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dcb2b600cf81ae1e49cdf5e78bbaaf030f72031ad6a107fefc0c3c3c65c59c04
+size 54740730

checkpoint-180/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae78313eb528c8d3695eebaf4de3539bd0a0bc6ee18c66af1ee183442f1758a0
+size 15024

checkpoint-180/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1b38031f60d9e88601d369ef46bcdcf2b5b03f2cb4ba93853bcb2328df7ebb7c
+size 15024

checkpoint-180/rng_state_2.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f58092375c93d237cd0e3149aecfbf83e2acdae46279e07a32920d01cb507e64
+size 15024

checkpoint-180/rng_state_3.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:83cd4bbff9962da7ec6787fcea8d65df7096917f9a5902e249ba7aee8887fe5f
+size 15024

checkpoint-180/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3db171fd1632e7fa1f2bf61bae0838c79d128613f85598160a41fd69ac877a77
+size 1064

checkpoint-180/special_tokens_map.json

@@ -0,0 +1,24 @@
+{
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "</s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "<unk>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-180/tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
+size 493443

checkpoint-180/tokenizer_config.json

@@ -0,0 +1,49 @@
+{
+  "add_bos_token": true,
+  "add_eos_token": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<unk>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "</s>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [],
+  "bos_token": "<s>",
+  "chat_template": "{% for message in messages %}{% if message['role'] == 'user' %}{{ bos_token + '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'system' %}{{ '<<SYS>>\\n' + message['content'] + '\\n<</SYS>>\\n\\n' }}{% elif message['role'] == 'assistant' %}{{ ' '  + message['content'] + ' ' + eos_token }}{% endif %}{% endfor %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "</s>",
+  "legacy": true,
+  "max_lenght": 8192,
+  "max_length": 8192,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "<unk>",
+  "padding": true,
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "LlamaTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false
+}

checkpoint-180/trainer_state.json

@@ -0,0 +1,387 @@
+{
+  "best_metric": 0.1363464593887329,
+  "best_model_checkpoint": "./mistral/21-04-24-Weni-WeniGPT-Agents-Mistral-1.0.6-SFT-1.0.10-DPO_Experiment on DPO with other hyperparameters and best SFT model of WeniGPT-2_max_steps-180_batch_8_2024-04-21_ppid_9/checkpoint-180",
+  "epoch": 5.806451612903226,
+  "eval_steps": 30,
+  "global_step": 180,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.3225806451612903,
+      "grad_norm": 34.21405792236328,
+      "learning_rate": 3.3333333333333333e-06,
+      "logits/chosen": -1.884800910949707,
+      "logits/rejected": -1.912030577659607,
+      "logps/chosen": -140.8845672607422,
+      "logps/rejected": -158.2601318359375,
+      "loss": 0.6927,
+      "rewards/accuracies": 0.15000000596046448,
+      "rewards/chosen": 0.0007378007285296917,
+      "rewards/margins": -0.0002584075555205345,
+      "rewards/rejected": 0.0009962082840502262,
+      "step": 10
+    },
+    {
+      "epoch": 0.6451612903225806,
+      "grad_norm": 27.711360931396484,
+      "learning_rate": 4.770114942528735e-06,
+      "logits/chosen": -1.7324174642562866,
+      "logits/rejected": -1.7914950847625732,
+      "logps/chosen": -173.26895141601562,
+      "logps/rejected": -220.0247039794922,
+      "loss": 0.612,
+      "rewards/accuracies": 0.75,
+      "rewards/chosen": 0.15256789326667786,
+      "rewards/margins": 0.1401277333498001,
+      "rewards/rejected": 0.012440156191587448,
+      "step": 20
+    },
+    {
+      "epoch": 0.967741935483871,
+      "grad_norm": 26.326221466064453,
+      "learning_rate": 4.482758620689656e-06,
+      "logits/chosen": -1.8204330205917358,
+      "logits/rejected": -1.8630510568618774,
+      "logps/chosen": -182.41458129882812,
+      "logps/rejected": -206.4346160888672,
+      "loss": 0.4701,
+      "rewards/accuracies": 0.8999999761581421,
+      "rewards/chosen": 0.6488608121871948,
+      "rewards/margins": 0.7796527743339539,
+      "rewards/rejected": -0.13079197704792023,
+      "step": 30
+    },
+    {
+      "epoch": 0.967741935483871,
+      "eval_logits/chosen": -1.7598947286605835,
+      "eval_logits/rejected": -1.805458903312683,
+      "eval_logps/chosen": -99.03199768066406,
+      "eval_logps/rejected": -156.521728515625,
+      "eval_loss": 0.4342271685600281,
+      "eval_rewards/accuracies": 0.8571428656578064,
+      "eval_rewards/chosen": 0.7998437285423279,
+      "eval_rewards/margins": 0.8333196043968201,
+      "eval_rewards/rejected": -0.033475928008556366,
+      "eval_runtime": 8.7814,
+      "eval_samples_per_second": 3.189,
+      "eval_steps_per_second": 0.797,
+      "step": 30
+    },
+    {
+      "epoch": 1.2903225806451613,
+      "grad_norm": 27.426502227783203,
+      "learning_rate": 4.1954022988505746e-06,
+      "logits/chosen": -1.7997443675994873,
+      "logits/rejected": -1.8489749431610107,
+      "logps/chosen": -143.94583129882812,
+      "logps/rejected": -212.5563201904297,
+      "loss": 0.3488,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 0.9175179600715637,
+      "rewards/margins": 1.094123125076294,
+      "rewards/rejected": -0.1766051948070526,
+      "step": 40
+    },
+    {
+      "epoch": 1.6129032258064515,
+      "grad_norm": 18.17804527282715,
+      "learning_rate": 3.908045977011495e-06,
+      "logits/chosen": -1.864464521408081,
+      "logits/rejected": -1.9360177516937256,
+      "logps/chosen": -191.2495880126953,
+      "logps/rejected": -250.14492797851562,
+      "loss": 0.2771,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 0.9626705050468445,
+      "rewards/margins": 1.180901288986206,
+      "rewards/rejected": -0.21823067963123322,
+      "step": 50
+    },
+    {
+      "epoch": 1.935483870967742,
+      "grad_norm": 16.31520652770996,
+      "learning_rate": 3.620689655172414e-06,
+      "logits/chosen": -1.8045103549957275,
+      "logits/rejected": -1.8450534343719482,
+      "logps/chosen": -169.26010131835938,
+      "logps/rejected": -184.78604125976562,
+      "loss": 0.2414,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 1.484909176826477,
+      "rewards/margins": 1.7093353271484375,
+      "rewards/rejected": -0.22442612051963806,
+      "step": 60
+    },
+    {
+      "epoch": 1.935483870967742,
+      "eval_logits/chosen": -1.7662041187286377,
+      "eval_logits/rejected": -1.813871145248413,
+      "eval_logps/chosen": -96.62606048583984,
+      "eval_logps/rejected": -156.85777282714844,
+      "eval_loss": 0.2903791069984436,
+      "eval_rewards/accuracies": 0.8571428656578064,
+      "eval_rewards/chosen": 1.5216225385665894,
+      "eval_rewards/margins": 1.655914068222046,
+      "eval_rewards/rejected": -0.13429169356822968,
+      "eval_runtime": 8.7812,
+      "eval_samples_per_second": 3.189,
+      "eval_steps_per_second": 0.797,
+      "step": 60
+    },
+    {
+      "epoch": 2.258064516129032,
+      "grad_norm": 13.167648315429688,
+      "learning_rate": 3.3333333333333333e-06,
+      "logits/chosen": -1.718693494796753,
+      "logits/rejected": -1.7614237070083618,
+      "logps/chosen": -130.20358276367188,
+      "logps/rejected": -175.7864990234375,
+      "loss": 0.1744,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.415719985961914,
+      "rewards/margins": 3.0347917079925537,
+      "rewards/rejected": -0.6190719604492188,
+      "step": 70
+    },
+    {
+      "epoch": 2.5806451612903225,
+      "grad_norm": 11.707513809204102,
+      "learning_rate": 3.0459770114942533e-06,
+      "logits/chosen": -1.7951810359954834,
+      "logits/rejected": -1.8545682430267334,
+      "logps/chosen": -151.82888793945312,
+      "logps/rejected": -208.30465698242188,
+      "loss": 0.1643,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.0986313819885254,
+      "rewards/margins": 2.8733813762664795,
+      "rewards/rejected": -0.7747501134872437,
+      "step": 80
+    },
+    {
+      "epoch": 2.903225806451613,
+      "grad_norm": 14.989990234375,
+      "learning_rate": 2.7586206896551725e-06,
+      "logits/chosen": -1.770471215248108,
+      "logits/rejected": -1.8411359786987305,
+      "logps/chosen": -139.56134033203125,
+      "logps/rejected": -230.6079559326172,
+      "loss": 0.1587,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.0346851348876953,
+      "rewards/margins": 3.044564723968506,
+      "rewards/rejected": -1.0098793506622314,
+      "step": 90
+    },
+    {
+      "epoch": 2.903225806451613,
+      "eval_logits/chosen": -1.7615691423416138,
+      "eval_logits/rejected": -1.8106638193130493,
+      "eval_logps/chosen": -95.37845611572266,
+      "eval_logps/rejected": -157.77059936523438,
+      "eval_loss": 0.2241911143064499,
+      "eval_rewards/accuracies": 0.8571428656578064,
+      "eval_rewards/chosen": 1.8959076404571533,
+      "eval_rewards/margins": 2.3040497303009033,
+      "eval_rewards/rejected": -0.4081418812274933,
+      "eval_runtime": 8.7827,
+      "eval_samples_per_second": 3.188,
+      "eval_steps_per_second": 0.797,
+      "step": 90
+    },
+    {
+      "epoch": 3.225806451612903,
+      "grad_norm": 11.479488372802734,
+      "learning_rate": 2.471264367816092e-06,
+      "logits/chosen": -1.7622451782226562,
+      "logits/rejected": -1.8047513961791992,
+      "logps/chosen": -161.26657104492188,
+      "logps/rejected": -206.4326934814453,
+      "loss": 0.1154,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 3.022019147872925,
+      "rewards/margins": 3.8095016479492188,
+      "rewards/rejected": -0.7874821424484253,
+      "step": 100
+    },
+    {
+      "epoch": 3.5483870967741935,
+      "grad_norm": 5.29118013381958,
+      "learning_rate": 2.1839080459770117e-06,
+      "logits/chosen": -1.808702826499939,
+      "logits/rejected": -1.8714277744293213,
+      "logps/chosen": -179.12928771972656,
+      "logps/rejected": -258.98846435546875,
+      "loss": 0.1145,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 1.8848085403442383,
+      "rewards/margins": 3.1940231323242188,
+      "rewards/rejected": -1.309214472770691,
+      "step": 110
+    },
+    {
+      "epoch": 3.870967741935484,
+      "grad_norm": 3.6187407970428467,
+      "learning_rate": 1.896551724137931e-06,
+      "logits/chosen": -1.729027509689331,
+      "logits/rejected": -1.7834007740020752,
+      "logps/chosen": -134.66946411132812,
+      "logps/rejected": -168.46185302734375,
+      "loss": 0.1243,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.2111563682556152,
+      "rewards/margins": 3.050034284591675,
+      "rewards/rejected": -0.8388776779174805,
+      "step": 120
+    },
+    {
+      "epoch": 3.870967741935484,
+      "eval_logits/chosen": -1.7586631774902344,
+      "eval_logits/rejected": -1.8082207441329956,
+      "eval_logps/chosen": -94.82170104980469,
+      "eval_logps/rejected": -158.57847595214844,
+      "eval_loss": 0.1827973872423172,
+      "eval_rewards/accuracies": 0.8571428656578064,
+      "eval_rewards/chosen": 2.062931776046753,
+      "eval_rewards/margins": 2.7134299278259277,
+      "eval_rewards/rejected": -0.65049809217453,
+      "eval_runtime": 8.7868,
+      "eval_samples_per_second": 3.187,
+      "eval_steps_per_second": 0.797,
+      "step": 120
+    },
+    {
+      "epoch": 4.193548387096774,
+      "grad_norm": 13.82438850402832,
+      "learning_rate": 1.6091954022988506e-06,
+      "logits/chosen": -1.8028233051300049,
+      "logits/rejected": -1.8499234914779663,
+      "logps/chosen": -142.3784637451172,
+      "logps/rejected": -213.98263549804688,
+      "loss": 0.0962,
+      "rewards/accuracies": 0.949999988079071,
+      "rewards/chosen": 2.311488151550293,
+      "rewards/margins": 3.0622975826263428,
+      "rewards/rejected": -0.7508091926574707,
+      "step": 130
+    },
+    {
+      "epoch": 4.516129032258064,
+      "grad_norm": 4.951977729797363,
+      "learning_rate": 1.3218390804597702e-06,
+      "logits/chosen": -1.7415574789047241,
+      "logits/rejected": -1.8030340671539307,
+      "logps/chosen": -117.66064453125,
+      "logps/rejected": -233.10983276367188,
+      "loss": 0.0862,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 1.8413982391357422,
+      "rewards/margins": 3.4544386863708496,
+      "rewards/rejected": -1.613040566444397,
+      "step": 140
+    },
+    {
+      "epoch": 4.838709677419355,
+      "grad_norm": 9.237640380859375,
+      "learning_rate": 1.0344827586206898e-06,
+      "logits/chosen": -1.7683824300765991,
+      "logits/rejected": -1.8145263195037842,
+      "logps/chosen": -156.50010681152344,
+      "logps/rejected": -191.0560302734375,
+      "loss": 0.0927,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.515368938446045,
+      "rewards/margins": 3.9733409881591797,
+      "rewards/rejected": -1.4579721689224243,
+      "step": 150
+    },
+    {
+      "epoch": 4.838709677419355,
+      "eval_logits/chosen": -1.757559895515442,
+      "eval_logits/rejected": -1.8073893785476685,
+      "eval_logps/chosen": -94.35372161865234,
+      "eval_logps/rejected": -159.0943603515625,
+      "eval_loss": 0.14616528153419495,
+      "eval_rewards/accuracies": 0.8571428656578064,
+      "eval_rewards/chosen": 2.2033259868621826,
+      "eval_rewards/margins": 3.008593797683716,
+      "eval_rewards/rejected": -0.805267333984375,
+      "eval_runtime": 8.7833,
+      "eval_samples_per_second": 3.188,
+      "eval_steps_per_second": 0.797,
+      "step": 150
+    },
+    {
+      "epoch": 5.161290322580645,
+      "grad_norm": 10.306894302368164,
+      "learning_rate": 7.471264367816093e-07,
+      "logits/chosen": -1.8050546646118164,
+      "logits/rejected": -1.8365217447280884,
+      "logps/chosen": -196.28428649902344,
+      "logps/rejected": -191.0373992919922,
+      "loss": 0.0554,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.9131319522857666,
+      "rewards/margins": 3.977607011795044,
+      "rewards/rejected": -1.0644750595092773,
+      "step": 160
+    },
+    {
+      "epoch": 5.483870967741936,
+      "grad_norm": 5.313375949859619,
+      "learning_rate": 4.5977011494252875e-07,
+      "logits/chosen": -1.7491801977157593,
+      "logits/rejected": -1.8197593688964844,
+      "logps/chosen": -149.62326049804688,
+      "logps/rejected": -245.06027221679688,
+      "loss": 0.0522,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.5887293815612793,
+      "rewards/margins": 4.40247106552124,
+      "rewards/rejected": -1.81374192237854,
+      "step": 170
+    },
+    {
+      "epoch": 5.806451612903226,
+      "grad_norm": 17.354381561279297,
+      "learning_rate": 1.7241379310344828e-07,
+      "logits/chosen": -1.8929452896118164,
+      "logits/rejected": -1.9294450283050537,
+      "logps/chosen": -174.45443725585938,
+      "logps/rejected": -209.23934936523438,
+      "loss": 0.0995,
+      "rewards/accuracies": 1.0,
+      "rewards/chosen": 2.020129442214966,
+      "rewards/margins": 3.534996747970581,
+      "rewards/rejected": -1.5148675441741943,
+      "step": 180
+    },
+    {
+      "epoch": 5.806451612903226,
+      "eval_logits/chosen": -1.7568327188491821,
+      "eval_logits/rejected": -1.8068283796310425,
+      "eval_logps/chosen": -94.06877899169922,
+      "eval_logps/rejected": -159.41087341308594,
+      "eval_loss": 0.1363464593887329,
+      "eval_rewards/accuracies": 0.8571428656578064,
+      "eval_rewards/chosen": 2.2888102531433105,
+      "eval_rewards/margins": 3.18902850151062,
+      "eval_rewards/rejected": -0.9002184867858887,
+      "eval_runtime": 8.7837,
+      "eval_samples_per_second": 3.188,
+      "eval_steps_per_second": 0.797,
+      "step": 180
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 180,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 6,
+  "save_steps": 90,
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-180/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a22ca11c84839a0a37f0bebcc44d22aa477fe426489192b0ba1ca8a27656d0d
+size 5432