---
library_name: transformers
license: apache-2.0
base_model: google-bert/bert-base-multilingual-cased
tags:
- generated_from_trainer
model-index:
- name: 66042c5d15c72970aacdd01c08465813
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# 66042c5d15c72970aacdd01c08465813

This model is a fine-tuned version of [google-bert/bert-base-multilingual-cased](https://huggingface.co/google-bert/bert-base-multilingual-cased) on the nyu-mll/glue [stsb] dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6360
- Data Size: 1.0
- Epoch Runtime: 10.9811
- Mse: 0.6361
- Mae: 0.6071
- R2: 0.7154

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- total_train_batch_size: 32
- total_eval_batch_size: 32
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: constant
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step | Validation Loss | Data Size | Epoch Runtime | Mse    | Mae    | R2      |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:-------------:|:------:|:------:|:-------:|
| No log        | 0     | 0    | 7.2959          | 0         | 1.2205        | 7.2971 | 2.2707 | -2.2643 |
| No log        | 1     | 179  | 3.8976          | 0.0078    | 1.5296        | 3.8987 | 1.6467 | -0.7440 |
| No log        | 2     | 358  | 2.2853          | 0.0156    | 1.6104        | 2.2860 | 1.2819 | -0.0226 |
| No log        | 3     | 537  | 2.5867          | 0.0312    | 2.0079        | 2.5877 | 1.3672 | -0.1576 |
| No log        | 4     | 716  | 2.1014          | 0.0625    | 2.3998        | 2.1022 | 1.2436 | 0.0596  |
| No log        | 5     | 895  | 0.8921          | 0.125     | 2.9282        | 0.8926 | 0.7631 | 0.6007  |
| 0.1315        | 6     | 1074 | 0.7837          | 0.25      | 4.1671        | 0.7842 | 0.6791 | 0.6492  |
| 0.7823        | 7     | 1253 | 0.7277          | 0.5       | 6.5455        | 0.7283 | 0.6781 | 0.6742  |
| 0.6355        | 8.0   | 1432 | 0.6166          | 1.0       | 11.2578       | 0.6170 | 0.6052 | 0.7240  |
| 0.3982        | 9.0   | 1611 | 0.6060          | 1.0       | 11.2043       | 0.6064 | 0.5842 | 0.7287  |
| 0.2705        | 10.0  | 1790 | 0.6099          | 1.0       | 11.5508       | 0.6101 | 0.6025 | 0.7271  |
| 0.2075        | 11.0  | 1969 | 0.5823          | 1.0       | 11.2443       | 0.5828 | 0.5789 | 0.7393  |
| 0.1817        | 12.0  | 2148 | 0.6110          | 1.0       | 10.8460       | 0.6114 | 0.5984 | 0.7265  |
| 0.1452        | 13.0  | 2327 | 0.5857          | 1.0       | 10.9459       | 0.5860 | 0.5731 | 0.7379  |
| 0.128         | 14.0  | 2506 | 0.5893          | 1.0       | 10.9863       | 0.5895 | 0.5831 | 0.7363  |
| 0.1102        | 15.0  | 2685 | 0.6360          | 1.0       | 10.9811       | 0.6361 | 0.6071 | 0.7154  |


### Framework versions

- Transformers 4.57.0
- Pytorch 2.8.0+cu128
- Datasets 4.3.0
- Tokenizers 0.22.1