The `S3RetryingInputStream` hides cases where S3 closes a connection partway through downloading a blob. By default it retries 3 times before failing the download. However, the number of failures tends to increase on larger blobs and often 3 failures is not enough to complete a multi-GB download if S3 is suffering from a cluster of failures as sometimes happens. Typically we make 10s-to-100s of MBs of progress between each failure even in this state.

I think we should increase the tenacity of `S3RetryingInputStream` when downloading larger blobs. For instance, we could not count a partial download towards the retry limit if it makes significant progress before failing.
modules/repository-s3/src/main/java/org/elasticsearch/repositories/s3/S3RetryingInputStream.java
throw addSuppressedExceptions(new NoSuchFileException("Blob object [" + blobKey + "] not found: " + e.getMessage()));
                }
            }
            throw addSuppressedExceptions(e);
        }
    }
    private long getStreamLength(final S3Object object) {
        final ObjectMetadata metadata = object.getObjectMetadata();
        try {
            
            final Long[] range = metadata.getContentRange();
            if (range != null) {
                assert range[1] >= range[0] : range[1] + " vs " + range[0];
                assert range[0] == start + currentOffset
                    : "Content-Range start value [" + range[0] + "] exceeds start [" + start + "] + current offset [" + currentOffset + ']';
                assert range[1] == end : "Content-Range end value [" + range[1] + "] exceeds end [" + end + ']';
                return range[1] - range[0] + 1L;
            }
            return metadata.getContentLength();
        } catch (Exception e) {
            assert false : e;
            return Long.MAX_VALUE - 1L; 
        }
    }
    @Override
    public int read() throws IOException {
        ensureOpen();
        while (true) {
            try {
                final int result = currentStream.read();