### Apache Airflow version

Other Airflow 2 version (please specify below)

### What happened

When running a DAG that is structured like:
```
@dag{dag_id="my_dag", default_args={"retries":0"}}
def dag():
    op = MyOperator.partial(task_id="my_task", retries=3).expand(...)
```

The following test fails:
```
    def test_retries(self) -> None:
        dag_bag = DagBag(dag_folder=DAG_FOLDER, include_examples=False)
        dag = dag_bag.dags["my_dag"]
        for task in dag.tasks:
            if "my_task" in task.task_id:
                self.assertEqual(3, task.retries) # fails - this is 0
```

When printing out `task.partial_kwargs`, and looking at how the default args and partial args are merged, it seems like the default args are always taking precedence, even though in the `partial` global function, the `retries` do get set later on with the task-level parameter value. This doesn't seem to be respected though.

### What you think should happen instead

_No response_

### How to reproduce

If you run my above unit test for a test DAG, on version 2.4.3, it should show up as a test failure.

### Operating System

OS Ventura

### Versions of Apache Airflow Providers

_No response_

### Deployment

Google Cloud Composer

### Deployment details

_No response_

### Anything else

_No response_

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

partial_kwargs.setdefault("resources", resources)
    partial_kwargs.setdefault("doc", doc)
    partial_kwargs.setdefault("doc_json", doc_json)
    partial_kwargs.setdefault("doc_md", doc_md)
    partial_kwargs.setdefault("doc_rst", doc_rst)
    partial_kwargs.setdefault("doc_yaml", doc_yaml)
    
    if "task_concurrency" in kwargs:  
        raise TypeError("unexpected argument: task_concurrency")
    if partial_kwargs["wait_for_downstream"]:
        partial_kwargs["depends_on_past"] = True
    partial_kwargs["start_date"] = timezone.convert_to_utc(partial_kwargs["start_date"])
    partial_kwargs["end_date"] = timezone.convert_to_utc(partial_kwargs["end_date"])
    if partial_kwargs["pool"] is None:
        partial_kwargs["pool"] = Pool.DEFAULT_POOL_NAME
    partial_kwargs["retries"] = parse_retries(partial_kwargs["retries"])
    partial_kwargs["retry_delay"] = coerce_timedelta(partial_kwargs["retry_delay"], key="retry_delay")
    if partial_kwargs["max_retry_delay"] is not None:
        partial_kwargs["max_retry_delay"] = coerce_timedelta(
            partial_kwargs["max_retry_delay"],
            key="max_retry_delay",
        )
    partial_kwargs["executor_config"] = partial_kwargs["executor_config"] or {}
    partial_kwargs["resources"] = coerce_resources(partial_kwargs["resources"])
    return OperatorPartial(
        operator_class=operator_class,
        kwargs=partial_kwargs,
        params=partial_params,
    )
class BaseOperatorMeta(abc.ABCMeta):