I've seen three builds fail now witht he same error and this started happening on Nov 7 so it's likely we've introduced something here:

https://gradle-enterprise.elastic.co/scans/failures?failures.failureClassification=all_failures&failures.failureMessage=Execution%20failed%20for%20task%20%27:x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0%23follower%23oneThirdUpgradedTest%27.%0A%3E%20%60cluster%7B:x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0-follower%7D%60%20failed%20to%20wait%20for%20cluster%20health%20yellow%20after%2040%20SECONDS%0A%20%20%20%20IO%20error%20while%20waiting%20cluster%0A%20%20%20%20%20%20503%20Service%20Unavailable&search.relativeStartTime=P28D&search.timeZoneId=America/Los_Angeles#

```
* What went wrong:
Execution failed for task ':x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0#follower#oneThirdUpgradedTest'.
> `cluster{:x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0-follower}` failed to wait for cluster health yellow after 40 SECONDS
    IO error while waiting cluster
      503 Service Unavailable
```

It seems the follower cluster is having issues coming back up after upgrade.
server/src/test/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceShardsAllocatorTests.java
"test",
                    ActionListener.wrap(
                        response -> listenerCalled.set(true),
                        exception -> { throw new AssertionError("should not happen in test", exception); }
                    )
                );
            }
            @Override
            public void onFailure(Exception e) {
                throw new AssertionError(e);
            }
        });
        deterministicTaskQueue.runAllTasks();
        try {
            assertTrue(listenerCalled.get());
            verifier.accept(clusterService.state().routingTable());
        } finally {
            clusterService.close();
        }
    }
    public void testCallListenersOnlyAfterProducingFreshInput() throws InterruptedException {
        var reconciliations = new AtomicInteger(0);
        var listenersCalled = new CountDownLatch(2);
        var clusterStateUpdatesExecuted = new CountDownLatch(2);
        var discoveryNode = createDiscoveryNode("node-0");
        var initialState = ClusterState.builder(ClusterName.DEFAULT)
            .nodes(DiscoveryNodes.builder().add(discoveryNode).localNodeId(discoveryNode.getId()).masterNodeId(discoveryNode.getId()))
            .build();
        var threadPool = new TestThreadPool(getTestName());
        var clusterService = ClusterServiceUtils.createClusterService(initialState, threadPool);