We limit the number of concurrent blob downloads per node and per recovery, but if a user configures too many concurrent recoveries they may run out of per-node download permits which yields the following message:

> Unable to acquire permit to use snapshot files during recovery, this recovery will recover index files from the source node. Ensure snapshot files can be used during recovery by setting [indices.recovery.max_concurrent_snapshot_file_downloads] to be no greater than [25]

The `[25]` here is wrong, the 25 permits are shared across all inbound recoveries so we should divide the suggested value by `cluster.routing.allocation.node_concurrent_incoming_recoveries`. In extreme cases the user may have configured so many concurrent recoveries that the only reasonable limit is zero, and in that case we should suggest setting `indices.recovery.use_snapshots: false` instead.


server/src/main/java/org/elasticsearch/indices/recovery/RecoverySettings.java
private void setUseSnapshotsDuringRecovery(boolean useSnapshotsDuringRecovery) {
        this.useSnapshotsDuringRecovery = useSnapshotsDuringRecovery;
    }
    public int getMaxConcurrentSnapshotFileDownloads() {
        return maxConcurrentSnapshotFileDownloads;
    }
    public void setMaxConcurrentSnapshotFileDownloads(int maxConcurrentSnapshotFileDownloads) {
        this.maxConcurrentSnapshotFileDownloads = maxConcurrentSnapshotFileDownloads;
    }
    private void setMaxConcurrentSnapshotFileDownloadsPerNode(int maxConcurrentSnapshotFileDownloadsPerNode) {
        this.maxConcurrentSnapshotFileDownloadsPerNode = maxConcurrentSnapshotFileDownloadsPerNode;
        this.maxSnapshotFileDownloadsPerNodeSemaphore.setMaxPermits(maxConcurrentSnapshotFileDownloadsPerNode);
    }
    @Nullable
    Releasable tryAcquireSnapshotDownloadPermits() {
        if (getUseSnapshotsDuringRecovery() == false) {
            return null;
        }
        final int maxConcurrentSnapshotFileDownloads = getMaxConcurrentSnapshotFileDownloads();
        final boolean permitAcquired = maxSnapshotFileDownloadsPerNodeSemaphore.tryAcquire(maxConcurrentSnapshotFileDownloads);
        if (permitAcquired == false) {
            logger.warn(
                String.format(
                    Locale.ROOT,
                    "Unable to acquire permit to use snapshot files during recovery, "
                        + "this recovery will recover index files from the source node. "
                        + "Ensure snapshot files can be used during recovery by setting [%s] to be no greater than [%d]",
                    INDICES_RECOVERY_MAX_CONCURRENT_SNAPSHOT_FILE_DOWNLOADS.getKey(),
                    this.maxConcurrentSnapshotFileDownloadsPerNode
                )