### Official Helm Chart version

1.7.0 (latest released)

### Apache Airflow version

2.5.1

### Kubernetes Version

1.24.6

### Helm Chart configuration

executor: "KubernetesExecutor"  # however same issue happens with LocalExecutor
logs:
  persistence:
    enabled: true
    size: 50Gi
    storageClassName: azurefile-csi

### Docker Image customizations

Using airflow-2.5.1-python3.10 as a base image.
Copy custom shared libraries into folder under /opt/airflow/company
Copy DAGs /opt/airflow/dags



### What happened

```console

After migrating from airflow 2.4.3 to 2.5.1 start getting error below. No other changes to custom image. No task is running because of this error:
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/__main__.py", line 39, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/cli_parser.py", line 52, in command
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line 108, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py", line 384, in task_run
    ti.init_run_context(raw=args.raw)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/models/taskinstance.py", line 2414, in init_run_context
    self._set_context(self)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/log/logging_mixin.py", line 77, in _set_context
    set_context(self.log, context)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/log/logging_mixin.py", line 213, in set_context
    flag = cast(FileTaskHandler, handler).set_context(value)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/log/file_task_handler.py", line 71, in set_context
    local_loc = self._init_file(ti)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/log/file_task_handler.py", line 382, in _init_file
    self._prepare_log_folder(Path(full_path).parent)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/log/file_task_handler.py", line 358, in _prepare_log_folder
    directory.chmod(mode)
  File "/usr/local/lib/python3.10/pathlib.py", line 1191, in chmod
    self._accessor.chmod(self, mode, follow_symlinks=follow_symlinks)
PermissionError: [Errno 1] Operation not permitted: '/opt/airflow/logs/dag_id=***/run_id=manual__2023-01-22T02:59:43.752407+00:00/task_id=***'
```

### What you think should happen instead

Seem like airflow attempts to set change log folder permissions and not permissioned to do it.
Getting same error when executing command manually (confirmed folder path exists): chmod 511 '/opt/airflow/logs/dag_id=***/run_id=manual__2023-01-22T02:59:43.752407+00:00/task_id=***'
chmod: changing permissions of '/opt/airflow/logs/dag_id=***/run_id=scheduled__2023-01-23T15:30:00+00:00/task_id=***': Operation not permitted



### How to reproduce

My understanding is that  this error happens before any custom code is executed. 

### Anything else

Error happens every time, unable to start any DAG while using airflow 2.5.1. Exactly same configuration works with 2.5.0 and 2.4.3.
Same image and configuration works fine while running locally using docker-composer.

### Are you willing to submit PR?

- [ ] Yes I am willing to submit a PR!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)

"""
        Prepare the log folder and ensure its mode is as configured.
        To handle log writing when tasks are impersonated, the log files need to
        be writable by the user that runs the Airflow command and the user
        that is impersonated. This is mainly to handle corner cases with the
        SubDagOperator. When the SubDagOperator is run, all of the operators
        run under the impersonated user and create appropriate log files
        as the impersonated user. However, if the user manually runs tasks
        of the SubDagOperator through the UI, then the log files are created
        by the user that runs the Airflow command. For example, the Airflow
        run command may be run by the `airflow_sudoable` user, but the Airflow
        tasks may be run by the `airflow` user. If the log files are not
        writable by both users, then it's possible that re-running a task
        via the UI (or vice versa) results in a permission error as the task
        tries to write to a log file created by the other user.
        We leave it up to the user to manage their permissions by exposing configuration for both
        new folders and new log files. Default is to make new log folders and files group-writeable
        to handle most common impersonation use cases. The requirement in this case will be to make
        sure that the same group is set as default group for both - impersonated user and main airflow
        user.
        """
        new_folder_permissions = int(
            conf.get("logging", "file_task_handler_new_folder_permissions", fallback="0o775"), 8
        )
        directory.mkdir(mode=new_folder_permissions, parents=True, exist_ok=True)
        if directory.stat().st_mode % 0o1000 != new_folder_permissions % 0o1000:
            print(f"Changing dir permission to {new_folder_permissions}")
            directory.chmod(new_folder_permissions)
    def _init_file(self, ti):