### Summary

Executing a playbook with many hosts, there is massive overhead in checking whether the hosts list is a pattern, see for yourself on this callgrind graph:

![grafik](https://user-images.githubusercontent.com/3154871/111359016-4a4bf480-868b-11eb-8c8f-55b6a6a10bd7.png)

https://github.com/ansible/ansible/blob/da60525610a384bb04833b1c6429d9db6a87ef64/lib/ansible/vars/manager.py#L490

That check has quadratic runtime with the number of hosts - with a lot of hosts, commenting that check out reduces runtime manyfold.

### Issue Type

Bug Report

### Component Name

variable manager

### Ansible Version

2.10.5, devel

### OS / Environment

Ubuntu

### Steps to Reproduce

```yaml
- hosts: localhost
  tasks:
    - set_fact:
        extra_tasks: true # compute it
        hosts: "{{ [an, array, with, 5100, computed, hosts] }}"

- hosts: "{{ hostvars.localhost.hosts }}"
  vars:
    extra_tasks: "{{ hostvars.localhost.do_extra_tasks }}"
  tasks:
   # some more tasks here
    - name: Execute extra tasks
      command: /bin/true # example
      when: extra_tasks
```

### Expected Results

Execute in tens of seconds.

### Actual Results

Take over 5 minutes to execute.
'''
        Sets or updates the given facts for a host in the fact cache.
        '''
        if not isinstance(facts, Mapping):
            raise AnsibleAssertionError("the type of 'facts' to set for nonpersistent_facts should be a Mapping but is a %s" % type(facts))
        try:
            self._nonpersistent_fact_cache[host].update(facts)
        except KeyError:
            self._nonpersistent_fact_cache[host] = facts
    def set_host_variable(self, host, varname, value):
        '''
        Sets a value in the vars_cache for a host.
        '''
        if host not in self._vars_cache:
            self._vars_cache[host] = dict()
        if varname in self._vars_cache[host] and isinstance(self._vars_cache[host][varname], MutableMapping) and isinstance(value, MutableMapping):
            self._vars_cache[host] = combine_vars(self._vars_cache[host], {varname: value})
        else:
            self._vars_cache[host][varname] = value
class VarsWithSources(MutableMapping):
    '''
    Dict-like class for vars that also provides source information for each var
    This class can only store the source for top-level vars. It does no tracking
    on its own, just shows a debug message with the information that it is provided
    when a particular var is accessed.
    '''
    def __init__(self, *args, **kwargs):