org.elasticsearch.indices.recovery.RecoveryTarget#restoreFileFromSnapshot does not check for cancellation during the download of a file. This means that if a recovery is cancelled mid-way, we will still not release the store/shard-lock until the file is fully downloaded.
For large files or slow downloads this can lead to problems when the shard is (re-)allocated to a node that is still executing such an interrupted recovery.

-> we need to add some checks for cancellation here like we have for other recovery modes.
server/src/main/java/org/elasticsearch/indices/recovery/RecoveryTarget.java
decRef();
            }
            try {
                newTargetCancellableThreads.execute(closedLatch::await);
            } catch (CancellableThreads.ExecutionCancelledException e) {
                logger.trace(
                    "new recovery target cancelled for shard {} while waiting on old recovery target with id [{}] to close",
                    shardId,
                    recoveryId
                );
                return false;
            }
            RecoveryState.Stage stage = indexShard.recoveryState().getStage();
            if (indexShard.recoveryState().getPrimary() && (stage == RecoveryState.Stage.FINALIZE || stage == RecoveryState.Stage.DONE)) {
                
                
                
                
                assert stage != RecoveryState.Stage.DONE : "recovery should not have completed when it's being reset";
                throw new IllegalStateException("cannot reset recovery as previous attempt made it past finalization step");
            }
            indexShard.performRecoveryRestart();
            return true;
        }
        return false;
    }
    /**
     * cancel the recovery. calling this method will clean temporary files and release the store
     * unless this object is in use (in which case it will be cleaned once all ongoing users call