**Build scan:**
https://gradle-enterprise.elastic.co/s/tzalctm47olle/tests/:server:test/org.elasticsearch.gateway.AsyncShardFetchTests/testTwoNodesRemoveOne

**Reproduction line:**
```
./gradlew ':server:test' --tests "org.elasticsearch.gateway.AsyncShardFetchTests.testTwoNodesRemoveOne" -Dtests.seed=9155673A51415810 -Dtests.locale=zh-TW -Dtests.timezone=Etc/GMT-6 -Druntime.java=17
```

**Applicable branches:**
main

**Reproduces locally?:**
No, however it seems to reproduce if executed in a cycle.
Additionally each test run lasts 10 seconds seemingly due to this line: 
https://github.com/elastic/elasticsearch/blob/8cd71706faa8c28b3bed69b9729a4bd1b6b7d346/server/src/test/java/org/elasticsearch/gateway/AsyncShardFetchTests.java#L474

**Failure history:**
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.gateway.AsyncShardFetchTests&tests.test=testTwoNodesRemoveOne

**Failure excerpt:**
```
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1719, name=elasticsearch[testTwoNodesRemoveOne][generic][T#3], state=RUNNABLE, group=TGRP-AsyncShardFetchTests]

  at __randomizedtesting.SeedInfo.seed([9155673A51415810:1672460D3C188CD8]:0)

  Caused by: java.lang.AssertionError: (No message provided)

    at __randomizedtesting.SeedInfo.seed([9155673A51415810]:0)
    at org.junit.Assert.fail(Assert.java:86)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at org.junit.Assert.assertTrue(Assert.java:52)
    at org.elasticsearch.gateway.AsyncShardFetchTests$TestFetch$1.lambda$run$0(AsyncShardFetchTests.java:474)
    at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1154)
    at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1127)
    at org.elasticsearch.gateway.AsyncShardFetchTests$TestFetch$1.run(AsyncShardFetchTests.java:474)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:891)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.lang.Thread.run(Thread.java:833)

```
server/src/test/java/org/elasticsearch/gateway/AsyncShardFetchTests.java
fetchData = test.fetchData(nodes, emptySet());
        assertThat(fetchData.hasData(), equalTo(true));
        assertThat(fetchData.getData().size(), equalTo(0));
    }
    public void testTwoNodesOnSetup() throws Exception {
        DiscoveryNodes nodes = DiscoveryNodes.builder().add(node1).add(node2).build();
        test.addSimulation(node1.getId(), response1);
        test.addSimulation(node2.getId(), response2);
        
        AsyncShardFetch.FetchResult<Response> fetchData = test.fetchData(nodes, emptySet());
        assertThat(test.getNumberOfInFlightFetches(), equalTo(2));
        assertThat(fetchData.hasData(), equalTo(false));
        assertThat(test.reroute.get(), equalTo(0));
        
        test.fireSimulationAndWait(node1.getId());
        
        assertThat(test.getNumberOfInFlightFetches(), equalTo(1));
        fetchData = test.fetchData(nodes, emptySet());
        assertThat(fetchData.hasData(), equalTo(false));
        
        test.fireSimulationAndWait(node2.getId());
        
        assertThat(test.reroute.get(), equalTo(2));
        fetchData = test.fetchData(nodes, emptySet());
        assertThat(test.getNumberOfInFlightFetches(), equalTo(0));
        assertThat(fetchData.hasData(), equalTo(true));
        assertThat(fetchData.getData().size(), equalTo(2));
        assertThat(fetchData.getData().get(node1), sameInstance(response1));
        assertThat(fetchData.getData().get(node2), sameInstance(response2));
    }