### CI Link

https://gradle-enterprise.elastic.co/s/7dleq35dp5stm

### Repro line

./gradlew ':x-pack:plugin:core:test' --tests "org.elasticsearch.xpack.core.ilm.SetSingleNodeAllocateStepTests.testPerformActionAttrsRequestFails" -Dtests.seed=6B438F3C3DADE5D2 -Dtests.locale=nl -Dtests.timezone=Asia/Chongqin

### Does it reproduce?

Yes

### Applicable branches

8.1, 8.0

### Failure history

Seems to fail rarely, but has failed before. Example: https://gradle-enterprise.elastic.co/s/d33b4fzn7yoh6

### Failure excerpt

```
org.elasticsearch.xpack.core.ilm.SetSingleNodeAllocateStepTests > testPerformActionAttrsRequestFails FAILED
    java.lang.AssertionError: Node roles must not be provided as attributes but saw attributes {ml=K}
        at __randomizedtesting.SeedInfo.seed([C679BACC1DA27FA7:CB16887322D9C088]:0)
        at org.elasticsearch.cluster.node.DiscoveryNode.<init>(DiscoveryNode.java:233)
        at org.elasticsearch.cluster.node.DiscoveryNode.<init>(DiscoveryNode.java:175)
```
x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/ilm/SetSingleNodeAllocateStepTests.java
.fPut(index.getName(), indexMetadata);
        IndexRoutingTable.Builder indexRoutingTable = IndexRoutingTable.builder(index)
            .addShard(TestShardRouting.newShardRouting(new ShardId(index, 0), "node_id_0", true, ShardRoutingState.STARTED));
        ClusterState clusterState = ClusterState.builder(ClusterState.EMPTY_STATE)
            .metadata(Metadata.builder().indices(indices.build()).transientSettings(clusterSettings))
            .nodes(nodes)
            .routingTable(RoutingTable.builder().add(indexRoutingTable).build())
            .build();
        SetSingleNodeAllocateStep step = createRandomInstance();
        expectThrows(
            NoNodeAvailableException.class,
            () -> PlainActionFuture.<Void, Exception>get(f -> step.performAction(indexMetadata, clusterState, null, f))
        );
        Mockito.verifyNoMoreInteractions(client);
    }
    public void testPerformActionAttrsNoNodesValid() {
        final int numNodes = randomIntBetween(1, 20);
        String[] validAttr = new String[] { "box_type", "valid" };
        String[] invalidAttr = new String[] { "box_type", "not_valid" };
        Settings.Builder indexSettings = settings(Version.CURRENT);
        indexSettings.put(IndexMetadata.INDEX_ROUTING_INCLUDE_GROUP_SETTING.getKey() + validAttr[0], validAttr[1]);
        IndexMetadata indexMetadata = IndexMetadata.builder(randomAlphaOfLength(10))
            .settings(indexSettings)
            .numberOfShards(randomIntBetween(1, 5))
            .numberOfReplicas(randomIntBetween(0, numNodes - 1))
            .build();
        Index index = indexMetadata.getIndex();
        Settings invalidNodeSettings = Settings.builder().put(Node.NODE_ATTRIBUTES.getKey() + invalidAttr[0], invalidAttr[1]).build();
        DiscoveryNodes.Builder nodes = DiscoveryNodes.builder();
        for (int i = 0; i < numNodes; i++) {