**Apache Airflow version**: 2.0.0a2

**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): v1.15.12

**Environment**:
- **OS** (e.g. from /etc/os-release): CentOS 7


**What happened**:

When using KubernetesExecutor, a single SIGTERM doesn't result in the scheduler actually exiting cleanly. However, it will exit after receiving a second SIGTERM.

The same behavior happens if you have an exception in the scheduler. An easy way to replicate it is to raise in `pod_mutation_hook`.

In practice, this means the scheduler doesn't get restarted by K8s after encountering a exception. Or, when terminating the scheduler pod, it sticks around until k8s ultimately sends it a SIGKILL after its grace period is up.

**What you expected to happen**:

The scheduler to exit after a single SIGTERM.

**How to reproduce it**:
The easiest way to reproduce it is to simply send a SIGKILL to your KubernetesExecutor scheduler pod:

```
$ kubectl exec {scheduler_pod_name} kill 1
```

Note, the pod stays running, while the logs report:

```
[2020-10-30 17:34:09,581] {scheduler_job.py:794} INFO - Exiting gracefully upon receiving signal 15
[2020-10-30 17:34:10,586] {process_utils.py:97} INFO - Sending Signals.SIGTERM to GPID 59
[2020-10-30 17:34:10,879] {process_utils.py:63} INFO - Process psutil.Process(pid=59, status='terminated', exitcode=0, started=
'17:33:23') (59) terminated with exit code 0
[2020-10-30 17:34:10,882] {process_utils.py:97} INFO - Sending Signals.SIGTERM to GPID 59
[2020-10-30 17:34:10,882] {scheduler_job.py:1329} INFO - Exited execute loop
```

Issue another `kill`:
```
$ kubectl exec {scheduler_pod_name} kill 1
```

And the pod will be restarted by k8s.

If you can't reproduce it, let me know and I'll try and get something running on kind.

**Anything else we need to know**:

The same behavior happens if you have an exception in the scheduler. An easy way to replicate it is to raise in a `pod_mutation_hook`.


airflow/jobs/scheduler_job.py
task_instance.pool_slots,
                        open_slots,
                        pool,
                    )
                    num_starving_tasks += 1
                    num_starving_tasks_total += 1
                    
                    continue
                executable_tis.append(task_instance)
                open_slots -= task_instance.pool_slots
                dag_concurrency_map[dag_id] += 1
                task_concurrency_map[(task_instance.dag_id, task_instance.task_id)] += 1
            Stats.gauge(f'pool.starving_tasks.{pool_name}', num_starving_tasks)
        Stats.gauge('scheduler.tasks.starving', num_starving_tasks_total)
        Stats.gauge('scheduler.tasks.running', num_tasks_in_executor)
        Stats.gauge('scheduler.tasks.executable', len(executable_tis))
        task_instance_str = "\n\t".join([repr(x) for x in executable_tis])
        self.log.info("Setting the following tasks to queued state:\n\t%s", task_instance_str)
        
        filter_for_tis = TI.filter_for_tis(executable_tis)
        session.query(TI).filter(filter_for_tis).update(
            
            
            {TI.state: State.QUEUED, TI.queued_dttm: timezone.utcnow(), TI.queued_by_job_id: self.id},
            synchronize_session=False,
        )
        for ti in executable_tis:
            make_transient(ti)
        return executable_tis
    def _enqueue_task_instances_with_queued_state(self, task_instances: List[TI]) -> None: