### Apache Airflow version

2.7.0

### What happened

Relates to investigation of #30327

When you define a DAG with a `schedule=datetime.timedelta(days=1)` and `catchup=False` and you manually trigger a run in the past (logical date in the past, e.g. a few days ago) then upon completion of the run another scheduled run is triggered - irrespective of a previous scheduled run was made just before (not a day ago)

### What you think should happen instead

A manual run should not trigger another run as the scheduling in general should follow the delta time definition of the DAG.

### How to reproduce

Use the following example DAG code and enable scheduling:
```
with DAG(
    dag_id="after_workday_delta_regression",
    start_date=pendulum.datetime(2023, 8, 1, tz="UTC"),
    catchup=False,
    schedule=datetime.timedelta(days=1),
    params={"test": 123}
):
    @task
    def test(ti: TaskInstance=None):
        print(ti.execution_date)
    test()
```

Wait a moment and see that one schedule is run automatically.
Then trigger a manual DAG run and set the execution date/logical date a few days into the past.
After execution you see a third run of the DAG just at the time of completion of the manual trigger as execution date. But the first execution was just a few moments before, not respecting the desired delta of 1 day.

### Operating System

Ubuntu 20.04 / Breeze Dev setup in Py 3.8 Container

### Versions of Apache Airflow Providers

not relevant

### Deployment

Other

### Deployment details

Started and tested with latest main branch and breeze.

### Anything else

I tried to find the root cause but was not able to locate it. Suspect it is rooted in scheduler_job_runner.py.
I'd supply a fix but was not able to fund the root cause. Expert knowledge is needed probably.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

datetime.timedelta(seconds=9),
                    tags={"dag_id": "test_start_dag_runs"},
                ),
            ]
        )
        assert dag.get_last_dagrun().creating_job_id == scheduler_job.id
    def test_extra_operator_links_not_loaded_in_scheduler_loop(self, dag_maker):
        """
        Test that Operator links are not loaded inside the Scheduling Loop (that does not include
        DagFileProcessorProcess) especially the critical loop of the Scheduler.
        This is to avoid running User code in the Scheduler and prevent any deadlocks
        """
        with dag_maker(dag_id="test_extra_operator_links_not_loaded_in_scheduler") as dag:
            
            _ = CustomOperator(task_id="custom_task")
        custom_task = dag.task_dict["custom_task"]
        
        assert custom_task.operator_extra_links
        session = settings.Session()
        scheduler_job = Job(executor=self.null_exec)
        self.job_runner = SchedulerJobRunner(job=scheduler_job)
        self.job_runner.processor_agent = mock.MagicMock()
        self.job_runner._start_queued_dagruns(session)
        session.flush()
        
        s_dag_2 = self.job_runner.dagbag.get_dag(dag.dag_id)
        custom_task = s_dag_2.task_dict["custom_task"]
        
        assert not custom_task.operator_extra_links
    def test_scheduler_create_dag_runs_does_not_raise_error(self, caplog, dag_maker):