### Elasticsearch Version

8.3.1

### Installed Plugins

_No response_

### Java Version

_bundled_

### OS Version

Linux 5.4.196-108.356.amzn2.x86_64

### Problem Description

Elasticsearch in EKS does not honour `eks.amazonaws.com/sts-regional-endpoints=true` for IAM access with AWS regions. It always sends it's API calls to `us-east-1`.

### Steps to Reproduce

Deploy Elasticsearch in EKS using operator with the following env
```
          containers:
            - env:
                - name: AWS_ROLE_SESSION_NAME
                  value: snapshot-repo
                - name: ZONE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.annotations['topology.kubernetes.io/zone']
```

Create service account with the annotations
```
  annotations:
    eks.amazonaws.com/role-arn: redacted
    eks.amazonaws.com/sts-regional-endpoints: "true"
```

Create snapshot repo
```
PUT _snapshot/s3backup
{
    "type": "s3",
    "settings": {
      "bucket": "redacted",
      "server_side_encryption": "true",
      "max_restore_bytes_per_sec": "500mb",
      "storage_class": "intelligent_tiering",
      "use_throttle_retries": "true",
      "readonly": "false",
      "base_path": "snapshot",
      "region": "us-west-2",
      "max_snapshot_bytes_per_sec": "500mb",
      "endpoint": "s3.us-west-2.amazonaws.com"
  }
}
```

Run snapshot

View cloudtrail events for service account name
![2022-08-08_11-36-48](https://user-images.githubusercontent.com/64858237/183456721-db86153c-ccc3-4821-8c84-f8c85cb4e495.png)


### Logs (if relevant)

_No response_
modules/repository-s3/src/main/java/org/elasticsearch/repositories/s3/S3Service.java
final TimeValue compareAndExchangeAntiContentionDelay;
    S3Service(Environment environment, Settings nodeSettings) {
        webIdentityTokenCredentialsProvider = new CustomWebIdentityTokenCredentialsProvider(
            environment,
            System::getenv,
            System::getProperty,
            Clock.systemUTC()
        );
        compareAndExchangeTimeToLive = REPOSITORY_S3_CAS_TTL_SETTING.get(nodeSettings);
        compareAndExchangeAntiContentionDelay = REPOSITORY_S3_CAS_ANTI_CONTENTION_DELAY_SETTING.get(nodeSettings);
    }
    /**
     * Refreshes the settings for the AmazonS3 clients and clears the cache of
     * existing clients. New clients will be build using these new settings. Old
     * clients are usable until released. On release they will be destroyed instead
     * of being returned to the cache.
     */
    public synchronized void refreshAndClearCache(Map<String, S3ClientSettings> clientsSettings) {
        
        
        releaseCachedClients();
        this.staticClientSettings = Maps.ofEntries(clientsSettings.entrySet());
        derivedClientSettings = emptyMap();
        assert this.staticClientSettings.containsKey("default") : "always at least have 'default'";
        
    }
    /**
     * Attempts to retrieve a client by its repository metadata and settings from the cache.
     * If the client does not exist it will be created.
     */