**Apache Airflow version**: 2.1.2


**Environment**:

- **Docker image**: official apache/airflow:2.1.2 extended with pip installed packages
- **Executor**: CeleryExecutor
- **Database**: PostgreSQL (engine version: 12.5)

**What happened**:

I've encountered `SerializedDagNotFound: DAG 'my_dag_id' not found in serialized_dag table` after deleting DAG via web UI. Exception is raised each time when UI is accessed. The exception itself does not impact UI, but still an event is sent to Sentry each time I interact with it.

**What you expected to happen**:

After DAG deletion I've expected that all records of it apart from logs would be deleted, but it's DAG run was still showing up in Webserver UI (even though, I couldn't find any records of DAG in the metadb, apart from dag_tag table still containing records related to deleted DAG (Which may be a reason for a new issue), but manual deletion of those records had no impact. 

After I've deleted DAG Run via Web UI, the exception is no longer raised.

**How to reproduce it**:

To this moment, I've only encountered this with one DAG, which was used for debugging purposes. It consists of multiple PythonOperators and it uses the same boilerplate code I use for dynamic DAG generation (return DAG object from `create_dag` function, add it to globals), but in it I've replaced dynamic generation logic with just `dag = create_dag(*my_args, **my_kwargs)`. I think this may have been caused by DAG run still running, when DAG was deleted, but cannot support this theory with actual information. 

<details>
  <summary>Traceback</summary> 

```python
SerializedDagNotFound: DAG 'my_dag_id' not found in serialized_dag table
  File "flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "flask/_compat.py", line 39, in reraise
    raise value
  File "flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "airflow/www/auth.py", line 34, in decorated
    return func(*args, **kwargs)
  File "airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "airflow/www/views.py", line 1679, in blocked
    dag = current_app.dag_bag.get_dag(dag_id)
  File "airflow/utils/session.py", line 70, in wrapper
    return func(*args, session=session, **kwargs)
  File "airflow/models/dagbag.py", line 186, in get_dag
    self._add_dag_from_db(dag_id=dag_id, session=session)
  File "airflow/models/dagbag.py", line 258, in _add_dag_from_db
    raise SerializedDagNotFound(f"DAG '{dag_id}' not found in serialized_dag table")
```
</details>

airflow/www/views.py
"""Shows all tries."""
        default_dag_run = conf.getint('webserver', 'default_dag_run_display_number')
        dag_id = request.args.get('dag_id')
        dag = current_app.dag_bag.get_dag(dag_id)
        base_date = request.args.get('base_date')
        num_runs = request.args.get('num_runs', default=default_dag_run, type=int)
        if base_date:
            base_date = timezone.parse(base_date)
        else:
            base_date = dag.get_latest_execution_date() or timezone.utcnow()
        root = request.args.get('root')
        if root:
            dag = dag.partial_subset(task_ids_or_regex=root, include_upstream=True, include_downstream=False)
        chart_height = wwwutils.get_chart_height(dag)
        chart = nvd3.lineChart(
            name="lineChart",
            x_is_date=True,
            y_axis_format='d',
            height=chart_height,
            chart_attr=self.line_chart_attr,
        )
        tis = dag.get_task_instances_before(base_date, num_runs, session=session)