### Description

Can we add the option to configure the Retry policy of the kubernetes CoreV1Api? Or set it to default have some more resilient configuration.

Today it appears to retry operations 3 times but with 0 backoff in between each try. Causing temporary network glitches to result in fatal errors.
Following the flow below:

1. `airflow.kubernetes.kube_client.get_kube_client()`
Calls `load_kube_config()` without any configuration set, this assigns a default configuration with `retries=None` to `CoreV1Api.set_default()`
1b.  Creates `CoreV1Api()` with `api_client=None`
1c. `ApiClient()` default constructor creates a default configuration object via `Configuration.get_default_copy(), this is the default injected above`
2. On request, through some complicated flow inside `ApiClient` and urllib3, this `configuration.retries` eventually finds its way into urllib `HTTPConnectionPool`, where if unset, it uses `urllib3.util.Retry.DEFAULT`, this has a policy of 3x retries with 0 backoff time in between. 
------

Configuring the ApiClient would mean changing the `get_kube_client()` to something roughly resembling:
```
client_config = Configuration()
client_config.retries = Retry(total=3, backoff=LOAD_FROM_CONFIG)
config.load_kube_config(...., client_configuration=client_config)
apiclient = ApiClient(client_config)
return CoreV1Api(apiclient)
```
I don't know myself how fine granularity is best to expose to be configurable from airflow. The retry object has a lot of different options, so do the rest of the kubernetes client Configuration object. Maybe it should be injected from a plugin rather than config-file? Maybe urllib or kubernets library have other ways to set default config?

### Use case/motivation

Our Kubernetes API server had some unknown hickup for 10 seconds, this caused the Airflow kubernetes executor to crash, restarting airflow and then it started killing pods that were running fine, showing following log: "Reset the following 1 orphaned TaskInstances"

If the retries would have had some backoff it would have likely survived this hickup.

See attachment for the full stack trace, it's too long to include inline. Here is the most interesting parts:

```
2022-06-29 21:25:49	Class={kubernetes_executor.py:111} Level=ERROR Unknown error in KubernetesJobWatcher. Failing
...
2022-06-29 21:25:49	Class={connectionpool.py:810} Level=WARNING Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fbe35de0c70>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/default/pods/REDACTED
2022-06-29 21:25:49	 urllib3.exceptions.ProtocolError: ("Connection broken: InvalidChunkLength(got length b'', 0 bytes read)", InvalidChunkLength(got length b'', 0 bytes read))
2022-06-29 21:25:49	Class={connectionpool.py:810} Level=WARNING Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fbe315ec040>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/default/pods/REDACTED
2022-06-29 21:25:49	Class={connectionpool.py:810} Level=WARNING Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fbe315ec670>: Failed to establish a new connection: [Errno 111] Connection refused')': /api/v1/namespaces/default/pods/REDACTED
...
2022-06-29 21:25:50	Class={kubernetes_executor.py:813} Level=INFO Shutting down Kubernetes executor
...
2022-06-29 21:26:08	Class={scheduler_job.py:696} Level=INFO Starting the scheduler
...
2022-06-29 21:27:29	Class={scheduler_job.py:1285} Level=INFO Message=Reset the following 1 orphaned TaskInstances:
```
[airflowkubernetsretrycrash.log](https://github.com/apache/airflow/files/9017815/airflowkubernetsretrycrash.log) 
From airflow version 2.3.2



### Related issues

_No response_

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

) -> client.CoreV1Api:
    """
    Retrieves Kubernetes client.
    :param in_cluster: whether we are in cluster
    :param cluster_context: context of the cluster
    :param config_file: configuration file
    :return kubernetes client
    :rtype client.CoreV1Api
    """
    if not has_kubernetes:
        raise _import_err
    if conf.getboolean("kubernetes_executor", "enable_tcp_keepalive"):
        _enable_tcp_keepalive()
    if in_cluster:
        config.load_incluster_config()
    else:
        if cluster_context is None:
            cluster_context = conf.get("kubernetes_executor", "cluster_context", fallback=None)
        if config_file is None:
            config_file = conf.get("kubernetes_executor", "config_file", fallback=None)
        config.load_kube_config(config_file=config_file, context=cluster_context)
    configuration = _get_default_configuration()
    if not conf.getboolean("kubernetes_executor", "verify_ssl"):
        configuration.verify_ssl = False
    ssl_ca_cert = conf.get("kubernetes_executor", "ssl_ca_cert")
    if ssl_ca_cert:
        configuration.ssl_ca_cert = ssl_ca_cert
    api_client = client.ApiClient(configuration=configuration)
    return client.CoreV1Api(api_client)