**Apache Airflow version**: 2.1.0

**What happened**:

We use a python script that reads JSON configs from db to dynamically generate DAGs. Sometimes, when JSON Configs is updated, we expect DAGs to be removed in Airflow.  This doesn't always happen. From my observation, one of the following happens:
1) DAG is removed because the python script no longer generates it.
2) DAG still exists but run history is empty. When trigger the DAG, first task is stuck in `queued` status indefinitely.

**What you expected to happen**:
I expect the right behavior should be 
>1) DAG is removed because the python script no longer generates it.

**How to reproduce it**:
* Create a python script in DAG folder that dynamically generates multiple DAGs
* Execute these dynamically generated DAGs a few times
* Use Airflow Variable to toggle (reduce) the # of DAGs generated
* Examine if number of dynamically generated DAGs in the web UI


airflow/models/dag.py
self.default_args['start_date'] = timezone.convert_to_utc(self.default_args['start_date'])
        if 'end_date' in self.default_args:
            self.default_args['end_date'] = timezone.convert_to_utc(self.default_args['end_date'])
        
        if timetable is None:
            self.timetable = create_timetable(schedule_interval, self.timezone)
            if schedule_interval is ScheduleIntervalArgNotSet:
                schedule_interval = DEFAULT_SCHEDULE_INTERVAL
            self.schedule_interval: ScheduleInterval = schedule_interval
        elif schedule_interval is ScheduleIntervalArgNotSet:
            self.timetable = timetable
            self.schedule_interval = self.timetable.summary
        else:
            raise TypeError("cannot specify both 'schedule_interval' and 'timetable'")
        if isinstance(template_searchpath, str):
            template_searchpath = [template_searchpath]
        self.template_searchpath = template_searchpath
        self.template_undefined = template_undefined
        self.parent_dag: Optional[DAG] = None  
        self.last_loaded = timezone.utcnow()
        self.safe_dag_id = dag_id.replace('.', '__dot__')
        self.max_active_runs = max_active_runs
        self.dagrun_timeout = dagrun_timeout
        self.sla_miss_callback = sla_miss_callback
        if default_view in DEFAULT_VIEW_PRESETS:
            self._default_view: str = default_view
        else:
            raise AirflowException(
                f'Invalid values of dag.default_view: only support '
                f'{DEFAULT_VIEW_PRESETS}, but get {default_view}'