I've seen three builds fail now witht he same error and this started happening on Nov 7 so it's likely we've introduced something here:

https://gradle-enterprise.elastic.co/scans/failures?failures.failureClassification=all_failures&failures.failureMessage=Execution%20failed%20for%20task%20%27:x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0%23follower%23oneThirdUpgradedTest%27.%0A%3E%20%60cluster%7B:x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0-follower%7D%60%20failed%20to%20wait%20for%20cluster%20health%20yellow%20after%2040%20SECONDS%0A%20%20%20%20IO%20error%20while%20waiting%20cluster%0A%20%20%20%20%20%20503%20Service%20Unavailable&search.relativeStartTime=P28D&search.timeZoneId=America/Los_Angeles#

```
* What went wrong:
Execution failed for task ':x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0#follower#oneThirdUpgradedTest'.
> `cluster{:x-pack:qa:rolling-upgrade-multi-cluster:v8.6.0-follower}` failed to wait for cluster health yellow after 40 SECONDS
    IO error while waiting cluster
      503 Service Unavailable
```

It seems the follower cluster is having issues coming back up after upgrade.
server/src/test/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceReconcilerTests.java
final var desiredBalance = desiredBalance(clusterState, (shardId, nodeId) -> true);
        final var allocationService = createTestAllocationService(
            routingAllocation -> reconcile(routingAllocation, desiredBalance),
            new SameShardAllocationDecider(settings, clusterSettings),
            new ReplicaAfterPrimaryActiveAllocationDecider(),
            new AllocationDecider() {
                @Override
                public Decision canAllocate(ShardRouting shardRouting, RoutingNode node, RoutingAllocation allocation) {
                    if (shardRouting.primary()) {
                        return Decision.YES;
                    } else {
                        
                        assert triedReplica.compareAndSet(false, true);
                        return replicaDecision;
                    }
                }
            }
        );
        var reroutedState = clusterState;
        boolean changed;
        do {
            triedReplica.set(false);
            final var newState = startInitializingShardsAndReroute(allocationService, reroutedState);
            changed = newState != reroutedState;
            reroutedState = newState;
        } while (changed);
        assertTrue(
            reroutedState.routingTable()
                .shardRoutingTable("index-0", 0)
                .replicaShards()