### CI Link

https://gradle-enterprise.elastic.co/s/yp56iltyyinha

### Repro line

./gradlew ':x-pack:plugin:ilm:qa:multi-node:javaRestTest' --tests "org.elasticsearch.xpack.ilm.ChangePolicyforIndexIT.testChangePolicyForIndex" -Dtests.seed=8825024EAE3089DD -Dtests.locale=en-PH -Dtests.timezone=Asia/Karachi -Druntime.java=16

### Does it reproduce?

No

### Applicable branches

7.17 (probably all 8+)

### Failure history

_No response_

### Failure excerpt

This is rather concerning
```
java.lang.AssertionError: policy step registry cache failed sanity check
	at org.elasticsearch.xpack.ilm.PolicyStepsRegistry.getStep(PolicyStepsRegistry.java:378) ~[?:?]
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.getCurrentStep(IndexLifecycleRunner.java:108) ~[?:?]
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPolicyAfterStateChange(IndexLifecycleRunner.java:430) ~[?:?]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:416) ~[?:?]
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.clusterChanged(IndexLifecycleService.java:319) ~[?:?]
	at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateListener(ClusterApplierService.java:573) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateListeners(ClusterApplierService.java:559) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:519) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:428) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.cluster.service.ClusterApplierService.access$000(ClusterApplierService.java:56) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:154) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:718) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:262) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:225) ~[elasticsearch-7.17.2-SNAPSHOT.jar:7.17.2-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]
	at java.lang.Thread.run(Thread.java:831) [?:?]
[2022-03-15T09:34:09.657115416Z] [BUILD] Stopping node
```
x-pack/plugin/ilm/src/test/java/org/elasticsearch/xpack/ilm/PolicyStepsRegistryTests.java
assertThat(((ShrinkStep) gotStep).getNumberOfShards(), equalTo(1));
        
        policyMap = Collections.singletonMap(
            updatedPolicy.getName(),
            new LifecyclePolicyMetadata(updatedPolicy, headers, randomNonNegativeLong(), randomNonNegativeLong())
        );
        lifecycleMetadata = new IndexLifecycleMetadata(policyMap, OperationMode.RUNNING);
        metadata = Metadata.builder(metadata).putCustom(IndexLifecycleMetadata.TYPE, lifecycleMetadata).build();
        try (XContentBuilder builder = JsonXContent.contentBuilder()) {
            builder.startObject();
            metadata.toXContent(builder, ToXContent.EMPTY_PARAMS);
            builder.endObject();
            logger.info("--> metadata: {}", Strings.toString(builder));
        }
        currentState = ClusterState.builder(ClusterName.DEFAULT).metadata(metadata).build();
        
        registry.update(currentState.metadata().custom(IndexLifecycleMetadata.TYPE));
        registeredStepsForPolicy = registry.getStepMap().get(newPolicy.getName());
        shrinkStep = registeredStepsForPolicy.entrySet()
            .stream()
            .filter(e -> e.getKey().getPhase().equals("warm") && e.getKey().getName().equals("shrink"))
            .findFirst()
            .get()
            .getValue();
        gotStep = registry.getStep(metadata.index(index), shrinkStep.getKey());
        assertThat(((ShrinkStep) shrinkStep).getNumberOfShards(), equalTo(2));
        assertThat(((ShrinkStep) gotStep).getNumberOfShards(), equalTo(1));
    }
}