We limit the number of concurrent blob downloads per node and per recovery, but if a user configures too many concurrent recoveries they may run out of per-node download permits which yields the following message:

> Unable to acquire permit to use snapshot files during recovery, this recovery will recover index files from the source node. Ensure snapshot files can be used during recovery by setting [indices.recovery.max_concurrent_snapshot_file_downloads] to be no greater than [25]

The `[25]` here is wrong, the 25 permits are shared across all inbound recoveries so we should divide the suggested value by `cluster.routing.allocation.node_concurrent_incoming_recoveries`. In extreme cases the user may have configured so many concurrent recoveries that the only reasonable limit is zero, and in that case we should suggest setting `indices.recovery.use_snapshots: false` instead.


server/src/test/java/org/elasticsearch/indices/recovery/RecoverySettingsTests.java
.withRandomMemory()
            .withNetworkBandwidth(ByteSizeValue.ofGb(between(1, 10)))
            .withDiskReadBandwidth(diskReadBandwidth)
            .withDiskWriteBandwidth(ByteSizeValue.ofMb(between(250, 500)))
            .withOperatorDefaultFactor(factor)
            .build();
        assertThat("Node bandwidth settings should all exist", recoverySettings.nodeBandwidthSettingsExist(), equalTo(true));
        assertThat(
            "Data node should use available disk read bandwidth",
            recoverySettings.getMaxBytesPerSec(),
            equalTo(
                ByteSizeValue.ofBytes(Math.round(Objects.requireNonNullElse(factor, DEFAULT_FACTOR_VALUE) * diskReadBandwidth.getBytes()))
            )
        );
        final ByteSizeValue diskWriteBandwidth = ByteSizeValue.ofMb(randomFrom(100, 250));
        recoverySettings = nodeRecoverySettings().withRoles(randomDataNodeRoles())
            .withRandomMemory()
            .withNetworkBandwidth(ByteSizeValue.ofGb(between(1, 10)))
            .withDiskReadBandwidth(ByteSizeValue.ofMb(between(250, 500)))
            .withDiskWriteBandwidth(diskWriteBandwidth)
            .withOperatorDefaultFactor(factor)
            .build();
        assertThat("Node bandwidth settings should all exist", recoverySettings.nodeBandwidthSettingsExist(), equalTo(true));
        assertThat(
            "Data node should use available disk write bandwidth",
            recoverySettings.getMaxBytesPerSec(),
            equalTo(
                ByteSizeValue.ofBytes(Math.round(Objects.requireNonNullElse(factor, DEFAULT_FACTOR_VALUE) * diskWriteBandwidth.getBytes()))
            )
        );