### Summary

Executing a playbook with many hosts, there is massive overhead in checking whether the hosts list is a pattern, see for yourself on this callgrind graph:

![grafik](https://user-images.githubusercontent.com/3154871/111359016-4a4bf480-868b-11eb-8c8f-55b6a6a10bd7.png)

https://github.com/ansible/ansible/blob/da60525610a384bb04833b1c6429d9db6a87ef64/lib/ansible/vars/manager.py#L490

That check has quadratic runtime with the number of hosts - with a lot of hosts, commenting that check out reduces runtime manyfold.

### Issue Type

Bug Report

### Component Name

variable manager

### Ansible Version

2.10.5, devel

### OS / Environment

Ubuntu

### Steps to Reproduce

```yaml
- hosts: localhost
  tasks:
    - set_fact:
        extra_tasks: true # compute it
        hosts: "{{ [an, array, with, 5100, computed, hosts] }}"

- hosts: "{{ hostvars.localhost.hosts }}"
  vars:
    extra_tasks: "{{ hostvars.localhost.do_extra_tasks }}"
  tasks:
   # some more tasks here
    - name: Execute extra tasks
      command: /bin/true # example
      when: extra_tasks
```

### Expected Results

Execute in tens of seconds.

### Actual Results

Take over 5 minutes to execute.
except KeyError:
                    iterator.get_next_task_for_host(self._inventory.get_host(host))
        
        
        failed_hosts = iterator.get_failed_hosts()
        unreachable_hosts = self._tqm._unreachable_hosts.keys()
        display.debug("running handlers")
        handler_result = self.run_handlers(iterator, play_context)
        if isinstance(handler_result, bool) and not handler_result:
            result |= self._tqm.RUN_ERROR
        elif not handler_result:
            result |= handler_result
        
        
        failed_hosts = set(failed_hosts).union(iterator.get_failed_hosts())
        unreachable_hosts = set(unreachable_hosts).union(self._tqm._unreachable_hosts.keys())
        
        if not isinstance(result, bool) and result != self._tqm.RUN_OK:
            return result
        elif len(unreachable_hosts) > 0:
            return self._tqm.RUN_UNREACHABLE_HOSTS
        elif len(failed_hosts) > 0:
            return self._tqm.RUN_FAILED_HOSTS
        else:
            return self._tqm.RUN_OK
    def get_hosts_remaining(self, play):
        self._set_hosts_cache(play, refresh=False)
        ignore = set(self._tqm._failed_hosts).union(self._tqm._unreachable_hosts)
        return [host for host in self._hosts_cache if host not in ignore]
    def get_failed_hosts(self, play):