### Apache Airflow version

2.2.3rc1 (release candidate)

### What happened

I ran a DAG that does nothing explicitly with context, so I'm surprised to see deprecation warnings about context being used (trimmed to cut down on noise):

```
baseoperator.py:1107 DeprecationWarning: Accessing 'execution_date' from the template is deprecated and will be removed in a future version. Please use 'data_interval_start' or 'logical_date' instead.
baseoperator.py:1107 DeprecationWarning: Accessing 'next_ds' from the template is deprecated and will be removed in a future version. Please use '{{ data_interval_end | ds }}' instead.
baseoperator.py:1107 DeprecationWarning: Accessing 'next_ds_nodash' from the template is deprecated and will be removed in a future version. Please use '{{ data_interval_end | ds_nodash }}' instead.
baseoperator.py:1107 DeprecationWarning: Accessing 'next_execution_date' from the template is deprecated and will be removed in a future version. Please use 'data_interval_end' instead.
baseoperator.py:1107 DeprecationWarning: Accessing 'prev_ds' from the template is deprecated and will be removed in a future version.
baseoperator.py:1107 DeprecationWarning: Accessing 'prev_ds_nodash' from the template is deprecated and will be removed in a future version.
baseoperator.py:1107 DeprecationWarning: Accessing 'prev_execution_date' from the template is deprecated and will be removed in a future version.
baseoperator.py:1107 DeprecationWarning: Accessing 'prev_execution_date_success' from the template is deprecated and will be removed in a future version. Please use 'prev_data_interval_start_success' instead.
baseoperator.py:1107 DeprecationWarning: Accessing 'tomorrow_ds' from the template is deprecated and will be removed in a future version.
baseoperator.py:1107 DeprecationWarning: Accessing 'tomorrow_ds_nodash' from the template is deprecated and will be removed in a future version.
baseoperator.py:1107 DeprecationWarning: Accessing 'yesterday_ds' from the template is deprecated and will be removed in a future version.
baseoperator.py:1107 DeprecationWarning: Accessing 'yesterday_ds_nodash' from the template is deprecated and will be removed in a future version.
```

### What you expected to happen

I'd only expect to see deprecation warnings about things my DAG is actually interacting with.

### How to reproduce

Run this DAG:

```python
from datetime import datetime

from airflow import DAG
from airflow.operators.bash import BashOperator

with DAG(
    dag_id="no-context", schedule_interval=None, start_date=datetime(2021, 12, 10)
) as dag:
    BashOperator(task_id="sleep", bash_command="echo 'Hello!'")
```

Note: I also tried a TaskFlow DAG and that wasn't impacted.

### Operating System

MacOS 11.6

### Versions of Apache Airflow Providers

_No response_

### Deployment

Virtualenv installation

### Deployment details

_No response_

### Anything else

This was introduced in #19886 (cc @uranusjr).

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

airflow/models/baseoperator.py
if isinstance(arg, XComArg):
                self.set_upstream(arg.operator)
            elif isinstance(arg, (tuple, set, list)):
                for elem in arg:
                    apply_set_upstream(elem)
            elif isinstance(arg, dict):
                for elem in arg.values():
                    apply_set_upstream(elem)
            elif hasattr(arg, "template_fields"):
                for elem in arg.template_fields:
                    apply_set_upstream(elem)
        for field in self.template_fields:
            if hasattr(self, field):
                arg = getattr(self, field)
                apply_set_upstream(arg)
    @property
    def priority_weight_total(self) -> int:
        """
        Total priority weight for the task. It might include all upstream or downstream tasks.
        depending on the weight rule.
          - WeightRule.ABSOLUTE - only own weight
          - WeightRule.DOWNSTREAM - adds priority weight of all downstream tasks
          - WeightRule.UPSTREAM - adds priority weight of all upstream tasks
        """
        if self.weight_rule == WeightRule.ABSOLUTE:
            return self.priority_weight
        elif self.weight_rule == WeightRule.DOWNSTREAM:
            upstream = False
        elif self.weight_rule == WeightRule.UPSTREAM: