**Apache Airflow version**: 2.0.0a2

**Kubernetes version (if you are using kubernetes)** (use `kubectl version`): v1.15.12

**Environment**:
- **OS** (e.g. from /etc/os-release): CentOS 7


**What happened**:

When using KubernetesExecutor, a single SIGTERM doesn't result in the scheduler actually exiting cleanly. However, it will exit after receiving a second SIGTERM.

The same behavior happens if you have an exception in the scheduler. An easy way to replicate it is to raise in `pod_mutation_hook`.

In practice, this means the scheduler doesn't get restarted by K8s after encountering a exception. Or, when terminating the scheduler pod, it sticks around until k8s ultimately sends it a SIGKILL after its grace period is up.

**What you expected to happen**:

The scheduler to exit after a single SIGTERM.

**How to reproduce it**:
The easiest way to reproduce it is to simply send a SIGKILL to your KubernetesExecutor scheduler pod:

```
$ kubectl exec {scheduler_pod_name} kill 1
```

Note, the pod stays running, while the logs report:

```
[2020-10-30 17:34:09,581] {scheduler_job.py:794} INFO - Exiting gracefully upon receiving signal 15
[2020-10-30 17:34:10,586] {process_utils.py:97} INFO - Sending Signals.SIGTERM to GPID 59
[2020-10-30 17:34:10,879] {process_utils.py:63} INFO - Process psutil.Process(pid=59, status='terminated', exitcode=0, started=
'17:33:23') (59) terminated with exit code 0
[2020-10-30 17:34:10,882] {process_utils.py:97} INFO - Sending Signals.SIGTERM to GPID 59
[2020-10-30 17:34:10,882] {scheduler_job.py:1329} INFO - Exited execute loop
```

Issue another `kill`:
```
$ kubectl exec {scheduler_pod_name} kill 1
```

And the pod will be restarted by k8s.

If you can't reproduce it, let me know and I'll try and get something running on kind.

**Anything else we need to know**:

The same behavior happens if you have an exception in the scheduler. An easy way to replicate it is to raise in a `pod_mutation_hook`.


tests/jobs/test_scheduler_job.py
dag_file = os.path.join(
            os.path.dirname(os.path.realpath(__file__)), '../dags/test_on_failure_callback.py'
        )
        dagbag = DagBag(dag_folder=dag_file, include_examples=False)
        dag_file_processor = DagFileProcessor(dag_ids=[], log=mock.MagicMock())
        with create_session() as session, NamedTemporaryFile(delete=False) as callback_file:
            session.query(TaskInstance).delete()
            dag = dagbag.get_dag('test_om_failure_callback_dag')
            task = dag.get_task(task_id='test_om_failure_callback_task')
            ti = TaskInstance(task, DEFAULT_DATE, State.RUNNING)
            session.add(ti)
            session.commit()
            requests = [
                TaskCallbackRequest(
                    full_filepath=dag.full_filepath,
                    simple_task_instance=SimpleTaskInstance(ti),
                    msg="Message",
                )
            ]
            callback_file.close()
            with mock.patch.dict("os.environ", {"AIRFLOW_CALLBACK_FILE": callback_file.name}):
                dag_file_processor.process_file(dag_file, requests)
            with open(callback_file.name) as callback_file2:
                content = callback_file2.read()
            assert "Callback fired" == content
            os.remove(callback_file.name)
    def test_should_mark_dummy_task_as_success(self):