### Apache Airflow version

main (development)

### What happened

Changing a non-mapped task with task instance note and task reschedule to a mapped task crashes scheduler when the task is cleared for rerun. Related commit where a similar fix was done.

commit a770edfac493f3972c10a43e45bcd0e7cfaea65f
Author: Ephraim Anierobi <splendidzigy24@gmail.com>
Date:   Mon Feb 20 20:45:25 2023 +0100

    Fix Scheduler crash when clear a previous run of a normal task that is now a mapped task (#29645)
    
    The fix was to clear the db references of the taskinstances in XCom, RenderedTaskInstanceFields
    and TaskFail. That way, we are able to run the mapped tasks

### What you think should happen instead

_No response_

### How to reproduce

1. Create below dag file with BashOperator non-mapped.
2. Schedule a dag run and wait for it to finish.
3. Add a task instance note to bash operator.
4. Change t1 to ` t1 = BashOperator.partial(task_id="bash").expand(bash_command=command)` and return `["echo hello"]` from get_command.
5. Restart the scheduler and clear the task. 
6. scheduler crashes on trying to use map_index though foreign key reference exists to task instance note and task reschedule.

```python
import datetime

from airflow.operators.bash import BashOperator
from airflow import DAG
from airflow.decorators import task

with DAG(dag_id="bash_simple", start_date=datetime.datetime(2022, 1, 1), schedule=None, catchup=False) as dag:

    @task
    def get_command(arg1, arg2):
        return "echo hello"

    command = get_command(1, 1)
    t1 = BashOperator(task_id="bash", bash_command=command)


if __name__ == '__main__':
    dag.test()

```

### Operating System

Ubuntu

### Versions of Apache Airflow Providers

_No response_

### Deployment

Virtualenv installation

### Deployment details

_No response_

### Anything else

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

print(2)
        @dag.task
        def second_task():
            print(2)
        third_task = EmptyOperator(task_id="third_task")
        forth_task = EmptyOperator(task_id="forth_task", on_success_callback=lambda x: print("hi"))
        first_task() >> [second_task(), third_task, forth_task]
        dag_run = dag_maker.create_dagrun()
        first_ti = dag_run.get_task_instance(task_id="first_task")
        second_ti = dag_run.get_task_instance(task_id="second_task")
        third_ti = dag_run.get_task_instance(task_id="third_task")
        forth_ti = dag_run.get_task_instance(task_id="forth_task")
        first_ti.state = State.SUCCESS
        second_ti.state = State.NONE
        third_ti.state = State.NONE
        forth_ti.state = State.NONE
        session.merge(first_ti)
        session.merge(second_ti)
        session.merge(third_ti)
        session.merge(forth_ti)
        session.commit()
        first_ti.schedule_downstream_tasks(session=session)
        second_task = dag_run.get_task_instance(task_id="second_task")
        third_task = dag_run.get_task_instance(task_id="third_task")
        forth_task = dag_run.get_task_instance(task_id="forth_task")
        assert second_task.state == State.SCHEDULED
        assert third_task.state == State.NONE
        assert forth_task.state == State.SCHEDULED
        assert "2 downstream tasks scheduled from follow-on schedule" in caplog.text
def test_mapped_task_expands_in_mini_scheduler_if_upstreams_are_done(dag_maker, caplog, session):