### Description

expanding tasks in batches to allow mapped tasks spawn more than 1024 processes.

### Use case/motivation

Maximum length of a list is limited to 1024 by `max_map_length (AIRFLOW__CORE__MAX_MAP_LENGTH)`.
during scheduling of the new tasks, an UPDATE query is ran that tries to set all the new tasks at once.  Increasing `max_map_length` more than 4K makes airflow scheduler completely unresponsive.

Also, Postgres throws `stack depth limit exceeded` error which can be fixed by updating to a newer version and setting `max_stack_depth` higher. But it doesn't really matter because airflow scheduler freezes up.
 
As a workaround, I split the dag runs into subdag runs which works but it would be much nicer if we didn't have to worry about exceeding  `max_map_length`. 

### Related issues

It was discussed here:
[Increasing 'max_map_length' leads to SQL 'max_stack_depth' error with 5000 dags to be spawned #28478](https://github.com/apache/airflow/discussions/28478)

### Are you willing to submit a PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

async_mode = not self.using_sqlite
        processor_timeout_seconds: int = conf.getint("core", "dag_file_processor_timeout")
        processor_timeout = timedelta(seconds=processor_timeout_seconds)
        if not self._standalone_dag_processor:
            self.processor_agent = DagFileProcessorAgent(
                dag_directory=Path(self.subdir),
                max_runs=self.num_times_parse_dags,
                processor_timeout=processor_timeout,
                dag_ids=[],
                pickle_dags=pickle_dags,
                async_mode=async_mode,
            )
        try:
            self.job.executor.job_id = self.job.id
            if self.processor_agent:
                self.log.debug("Using PipeCallbackSink as callback sink.")
                self.job.executor.callback_sink = PipeCallbackSink(
                    get_sink_pipe=self.processor_agent.get_callbacks_pipe
                )
            else:
                from airflow.callbacks.database_callback_sink import DatabaseCallbackSink
                self.log.debug("Using DatabaseCallbackSink as callback sink.")
                self.job.executor.callback_sink = DatabaseCallbackSink()
            self.job.executor.start()
            self.register_signals()
            if self.processor_agent:
                self.processor_agent.start()
            execute_start_time = timezone.utcnow()