**Apache Airflow version**: 2.1.0

**What happened**:

We use a python script that reads JSON configs from db to dynamically generate DAGs. Sometimes, when JSON Configs is updated, we expect DAGs to be removed in Airflow.  This doesn't always happen. From my observation, one of the following happens:
1) DAG is removed because the python script no longer generates it.
2) DAG still exists but run history is empty. When trigger the DAG, first task is stuck in `queued` status indefinitely.

**What you expected to happen**:
I expect the right behavior should be 
>1) DAG is removed because the python script no longer generates it.

**How to reproduce it**:
* Create a python script in DAG folder that dynamically generates multiple DAGs
* Execute these dynamically generated DAGs a few times
* Use Airflow Variable to toggle (reduce) the # of DAGs generated
* Examine if number of dynamically generated DAGs in the web UI


airflow/models/dag.py
) -> List[TaskInstance]:
        """
        Set the state of a TaskInstance to the given state, and clear its downstream tasks that are
        in failed or upstream_failed state.
        :param task_id: Task ID of the TaskInstance
        :type task_id: str
        :param execution_date: execution_date of the TaskInstance
        :type execution_date: datetime
        :param state: State to set the TaskInstance to
        :type state: State
        :param upstream: Include all upstream tasks of the given task_id
        :type upstream: bool
        :param downstream: Include all downstream tasks of the given task_id
        :type downstream: bool
        :param future: Include all future TaskInstances of the given task_id
        :type future: bool
        :param commit: Commit changes
        :type commit: bool
        :param past: Include all past TaskInstances of the given task_id
        :type past: bool
        """
        from airflow.api.common.experimental.mark_tasks import set_state
        task = self.get_task(task_id)
        task.dag = self
        altered = set_state(
            tasks=[task],
            execution_date=execution_date,
            upstream=upstream,
            downstream=downstream,
            future=future,