### Summary

When using `set_fact` in combination with `delegate_to`, `delegate_facts` and `when` it seems to actually be only run on the first host: 

```yaml
- hosts: localhost
  gather_facts: no
  connection: local
  tasks:
    - set_fact:
        test: 123
      delegate_to: "{{ item }}"
      delegate_facts: true
      when: test is not defined
      loop: "{{ groups['all'] }}"
      # note: the following will STILL fail!
      #loop: "{{ groups['all'] | difference(["localhost"]) }}"
```

The problem only arises when setting the variable that is actually used in the `when` clause. If I had to guess, I assume the `when` is checked in the wrong loop iteration. Setting a fact that is not used in `when` will work fine.

#20508 seems to be related in some regard but was fixed some time ago.

### Issue Type

Bug Report

### Component Name

set_fact

### Ansible Version

```console
$ ansible --version
ansible [core 2.11.5]
  config file = /Users/hojerst/.ansible.cfg
  configured module search path = ['/Users/hojerst/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/hojerst/.pyenv/versions/3.9.4/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/hojerst/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/hojerst/.pyenv/versions/3.9.4/bin/ansible
  python version = 3.9.4 (default, Apr 11 2021, 17:03:42) [Clang 12.0.0 (clang-1200.0.32.29)]
  jinja version = 2.11.3
  libyaml = True
```


### Configuration

```console
$ ansible-config dump --only-changed

DEFAULT_VAULT_PASSWORD_FILE(/Users/hojerst/.ansible.cfg) = /Users/hojerst/.ansible/vault-pass%
```


### OS / Environment

MacOS 12.0 Beta (21A5506j)

### Steps to Reproduce

<!--- Paste example playbooks or commands between quotes below -->
```yaml
- hosts: localhost
  gather_facts: no
  connection: local
  tasks:
    - set_fact:
        test: 123
      delegate_to: "{{ item }}"
      delegate_facts: true
      when: test is not defined
      loop: "{{ groups['all'] | difference(['localhost']) }}"

    - debug:
        var: test
```

```sh
ansible-playbook playbook.yml -i host1,host2,host3
```


### Expected Results

PLAY [localhost] **************************************************************************************************************

TASK [set_fact] ***************************************************************************************************************
ok: [localhost -> host1] => (item=host1)
ok: [localhost -> host2] => (item=host2)
ok: [localhost -> host3] => (item=host3)

PLAY RECAP ********************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0


### Actual Results

```console

PLAY [localhost] **************************************************************************************************************

TASK [set_fact] ***************************************************************************************************************
ok: [localhost -> host1] => (item=host1)
skipping: [localhost] => (item=host2)
skipping: [localhost] => (item=host3)

TASK [debug] ******************************************************************************************************************
ok: [localhost] => {
    "test": "VARIABLE IS NOT DEFINED!"
}

PLAY RECAP ********************************************************************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

```


### Code of Conduct

- [X] I agree to follow the Ansible Code of Conduct
if self._task.changed_when is not None and self._task.changed_when:
                    cond = Conditional(loader=self._loader)
                    cond.when = self._task.changed_when
                    result['changed'] = cond.evaluate_conditional(templar, vars_copy)
            def _evaluate_failed_when_result(result):
                if self._task.failed_when:
                    cond = Conditional(loader=self._loader)
                    cond.when = self._task.failed_when
                    failed_when_result = cond.evaluate_conditional(templar, vars_copy)
                    result['failed_when_result'] = result['failed'] = failed_when_result
                else:
                    failed_when_result = False
                return failed_when_result
            if 'ansible_facts' in result and self._task.action not in C._ACTION_DEBUG:
                if self._task.action in C._ACTION_WITH_CLEAN_FACTS:
                    vars_copy.update(result['ansible_facts'])
                else:
                    
                    af = wrap_var(result['ansible_facts'])
                    vars_copy['ansible_facts'] = combine_vars(vars_copy.get('ansible_facts', {}), namespace_facts(af))
                    if C.INJECT_FACTS_AS_VARS:
                        vars_copy.update(clean_facts(af))
            
            if 'failed' not in result:
                
                if 'rc' in result and result['rc'] not in [0, "0"]: