### Summary

`ansible-galaxy collection install` makes a lot of requests to galaxy.ansible.com and can occasionally hit the rate limit. When that happens, galaxy.ansible.com will return a 520 http code and `ansible-galaxy` fails to download the collection and exits. Ideally when the client encounters a rate limiting http code (either 520, or 429), it should wait, slow down the request rate and try again rather than exiting.

More information is available here: https://github.com/ansible/galaxy/issues/2429 

### Issue Type

Bug Report

### Component Name

ansible-galaxy

### Ansible Version

```console
$ ansible --version
root@ubuntu-s-1vcpu-1gb-nyc3-01:~# ansible --version
ansible 2.10.7
  config file = None
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]
```


### Configuration

```console
$ ansible-config dump --only-changed
root@ubuntu-s-1vcpu-1gb-nyc3-01:~# ansible-config dump --only-changed
root@ubuntu-s-1vcpu-1gb-nyc3-01:~#
```


### OS / Environment

All

### Steps to Reproduce

Run `ansible-galaxy collection install `amazon.aws`. If your internet is fast enough, this will occasionally fail when galaxy.ansible.com returns a 520 error code.

### Expected Results

Collection should be installed.

### Actual Results

```console
`ansible-galaxy` encounters a 429 or 520 http code and exits.
```


### Code of Conduct

I agree to follow the Ansible Code of Conduct


path_cache = self._cache[cache_id][url_info.path]
            
            paginated_key = None
            for key in ['data', 'results']:
                if key in data:
                    paginated_key = key
                    break
            if paginated_key:
                path_cache['paginated'] = True
                results = path_cache.setdefault('results', [])
                for result in data[paginated_key]:
                    results.append(result)
            else:
                path_cache['results'] = data
        return data
    def _add_auth_token(self, headers, url, token_type=None, required=False):
        
        if 'Authorization' in headers:
            return
        if not self.token and required:
            raise AnsibleError("No access token or username set. A token can be set with --api-key "
                               "or at {0}.".format(to_native(C.GALAXY_TOKEN_PATH)))
        if self.token:
            headers.update(self.token.headers())
    @cache_lock
    def _set_cache(self):
        with open(self._b_cache_path, mode='wb') as fd:
            fd.write(to_bytes(json.dumps(self._cache), errors='surrogate_or_strict'))
    @g_connect(['v1'])
    def authenticate(self, github_token):