Given a cluster with two hot nodes:

```
GET _cat/nodes?s=name
10.42.8.0    66 63 0 2.74 1.59 1.27 himrst * instance-0000000000
10.42.10.202 26 63 4 0.79 0.79 0.97 himrst - instance-0000000001
10.42.13.123 57 97 0 3.18 3.34 3.44 mv     - tiebreaker-0000000002
```

I create an index with a `_tier_preference` like this:

```
PUT tweets-1
{
  "settings": {
    "index.routing.allocation.include._tier_preference": "data_warm,data_hot"
  }
}
```

The shards are allocated to the hot nodes (they would love most to be allocated to warm nodes, but there aren't any, luckily they're not picky):

```
GET _cat/shards/tweets*?s=index,shard,prirep
tweets-1 0 p STARTED 0 225b 10.42.10.202 instance-0000000001
tweets-1 0 r STARTED 0 225b 10.42.8.0    instance-0000000000
```

I size up my cluster, adding some warm nodes. It now looks like this:

```
GET _cat/nodes?s=name
10.42.8.0    70 63 2 1.79 1.41 1.25 himrst * instance-0000000000
10.42.10.202 20 63 2 1.81 0.99 0.95 himrst - instance-0000000001
10.42.4.1    10 61 1 1.24 0.86 0.71 rw     - instance-0000000003
10.42.255.41 38 61 1 2.32 2.61 2.46 rw     - instance-0000000004
10.42.13.123 36 97 0 1.89 2.88 3.21 mv     - tiebreaker-0000000002
```

My shards have also moved themselves onto the new warm nodes, in accordance with their `_tier_preference`[:](https://camo.githubusercontent.com/f112d98914158f9c0c2b94f9aeeab2739f040f86faae45262ab426f018dd156a/68747470733a2f2f7777772e6d656d6563726561746f722e636f6d2f7374617469632f696d616765732f6d656d65732f3335303837302e6a7067)

```
GET _cat/nodes?s=name
tweets-1 0 p STARTED 0 247b 10.42.255.41 instance-0000000004
tweets-1 0 r STARTED 0 225b 10.42.4.1    instance-0000000003
```

Time passes, and I realize I'm not using my warm nodes as much as I'd planned to, so I issue two [`shutdown`](https://www.elastic.co/guide/en/elasticsearch/reference/8.8/put-shutdown.html) requests to tell the cluster that the warm nodes are going to go away. Those requests probably look a lot like this, but please don't get too hung up on this particular detail:

```
PUT /_nodes/_6IelXFtSPqIX7u3jfOFVQ/shutdown
{
  "type": "remove", 
  "reason": "..."
}

PUT /_nodes/F77A6HAPTdKTu0ImysCxiA/shutdown
{
  "type": "remove", 
  "reason": "..."
}
```

Given that Elasticsearch was able to automatically expand from hot-only to hot and warm, my desire as a user of Elasticsearch is that the same flexibility would be applied in reverse when I want to shrink my cluster. However, it doesn't seem like that's the case (unfortunately). Let's look at `GET _cluster/allocation/explain` to see why.

Here's my request:

```
GET _cluster/allocation/explain
{
  "index": "tweets-1",
  "shard": 0,
  "primary": true
}
```

And here's the response:

```
{
  "index": "tweets-1",
  "shard": 0,
  "primary": true,
  "current_state": "started",
  "current_node": {
    "id": "F77A6HAPTdKTu0ImysCxiA",
    "name": "instance-0000000004",
    "transport_address": "10.42.255.41:19994",
    "attributes": {
      "data": "warm",
      "xpack.installed": "true",
      "logical_availability_zone": "zone-1",
      "availability_zone": "us-central1-c",
      "region": "unknown-region",
      "instance_configuration": "gcp.es.datawarm.n2.68x10x190",
      "server_name": "instance-0000000004.0987d9fba82e49d08dd780dec13b6798"
    }
  },
  "can_remain_on_current_node": "no",
  "can_remain_decisions": [
    {
      "decider": "node_shutdown",
      "decision": "NO",
      "explanation": "node [F77A6HAPTdKTu0ImysCxiA] is preparing to be removed from the cluster"
    }
  ],
  "can_move_to_other_node": "no",
  "move_explanation": "This shard may not remain on its current node, but Elasticsearch isn't allowed to move it to another node. Choose a node to which you expect this shard to be allocated, find this node in the node-by-node explanation, and address the reasons which prevent Elasticsearch from allocating this shard there.",
  "node_allocation_decisions": [
    {
      "node_id": "_6IelXFtSPqIX7u3jfOFVQ",
      "node_name": "instance-0000000003",
      "transport_address": "10.42.4.1:19849",
      "node_attributes": {
        "data": "warm",
        "xpack.installed": "true",
        "logical_availability_zone": "zone-0",
        "availability_zone": "us-central1-a",
        "region": "unknown-region",
        "instance_configuration": "gcp.es.datawarm.n2.68x10x190",
        "server_name": "instance-0000000003.0987d9fba82e49d08dd780dec13b6798"
      },
      "node_decision": "no",
      "weight_ranking": 1,
      "deciders": [
        {
          "decider": "node_shutdown",
          "decision": "NO",
          "explanation": "node [_6IelXFtSPqIX7u3jfOFVQ] is preparing to be removed from the cluster"
        },
        {
          "decider": "same_shard",
          "decision": "NO",
          "explanation": "a copy of this shard is already allocated to this node [[tweets-1][0], node[_6IelXFtSPqIX7u3jfOFVQ], [R], s[STARTED], a[id=pVXsE91AQRCBUGy8j_fE1w], failed_attempts[0]]"
        },
        {
          "decider": "awareness",
          "decision": "NO",
          "explanation": "there are [2] copies of this shard and [2] values for attribute [logical_availability_zone] ([zone-0, zone-1] from nodes in the cluster and no forced awareness) so there may be at most [1] copies of this shard allocated to nodes with each value, but (including this copy) there would be [2] copies allocated to nodes with [node.attr.logical_availability_zone: zone-0]"
        }
      ]
    },
    {
      "node_id": "EVzA2lslS1eNPFoVEO3tJQ",
      "node_name": "instance-0000000001",
      "transport_address": "10.42.10.202:19393",
      "node_attributes": {
        "data": "hot",
        "xpack.installed": "true",
        "logical_availability_zone": "zone-1",
        "availability_zone": "us-central1-c",
        "region": "unknown-region",
        "instance_configuration": "gcp.es.datahot.n2.68x10x45",
        "server_name": "instance-0000000001.0987d9fba82e49d08dd780dec13b6798"
      },
      "node_decision": "no",
      "weight_ranking": 2,
      "deciders": [
        {
          "decider": "data_tier",
          "decision": "NO",
          "explanation": "index has a preference for tiers [data_warm,data_hot] and node does not meet the required [data_warm] tier"
        }
      ]
    },
    {
      "node_id": "FMcDYr1GTqWsji0KeR1s6A",
      "node_name": "instance-0000000000",
      "transport_address": "10.42.8.0:19993",
      "node_attributes": {
        "data": "hot",
        "xpack.installed": "true",
        "logical_availability_zone": "zone-0",
        "availability_zone": "us-central1-b",
        "region": "unknown-region",
        "instance_configuration": "gcp.es.datahot.n2.68x10x45",
        "server_name": "instance-0000000000.0987d9fba82e49d08dd780dec13b6798"
      },
      "node_decision": "no",
      "weight_ranking": 3,
      "deciders": [
        {
          "decider": "awareness",
          "decision": "NO",
          "explanation": "there are [2] copies of this shard and [2] values for attribute [logical_availability_zone] ([zone-0, zone-1] from nodes in the cluster and no forced awareness) so there may be at most [1] copies of this shard allocated to nodes with each value, but (including this copy) there would be [2] copies allocated to nodes with [node.attr.logical_availability_zone: zone-0]"
        },
        {
          "decider": "data_tier",
          "decision": "NO",
          "explanation": "index has a preference for tiers [data_warm,data_hot] and node does not meet the required [data_warm] tier"
        }
      ]
    }
  ]
}
```

That's a bit of a json splat, so I'll sum up the decisions for you:

| node_name | node_id | hot/warm | outcome
| --- | --- | --- | --- |
| `instance-0000000000` | `FMcDYr1GTqWsji0KeR1s6A` | hot | `"[...] there would be [2] copies allocated to nodes with [node.attr.logical_availability_zone: zone-0]` (okay, yeah, I see this one) |
| `instance-0000000001` | `EVzA2lslS1eNPFoVEO3tJQ`  | hot |  Wait a minute! This is where it could go, one would think! See below. |
| `instance-0000000003` | `_6IelXFtSPqIX7u3jfOFVQ` | warm | `"explanation": "node [_6IelXFtSPqIX7u3jfOFVQ] is preparing to be removed from the cluster"` (fair, I agree completely) |
| `instance-0000000004` | `F77A6HAPTdKTu0ImysCxiA` | warm | `"can_remain_on_current_node": "no",` (also fair, also agree) |

So the crucial decision is related to `instance-0000000001`, and the decision there is:

```
    {
      "node_id": "EVzA2lslS1eNPFoVEO3tJQ",
      "node_name": "instance-0000000001",
      "transport_address": "10.42.10.202:19393",
      "node_attributes": {
        "data": "hot",
        "xpack.installed": "true",
        "logical_availability_zone": "zone-1",
        "availability_zone": "us-central1-c",
        "region": "unknown-region",
        "instance_configuration": "gcp.es.datahot.n2.68x10x45",
        "server_name": "instance-0000000001.0987d9fba82e49d08dd780dec13b6798"
      },
      "node_decision": "no",
      "weight_ranking": 2,
      "deciders": [
        {
          "decider": "data_tier",
          "decision": "NO",
          "explanation": "index has a preference for tiers [data_warm,data_hot] and node does not meet the required [data_warm] tier"
        }
      ]
    },
```

Which is to say, the DataTierAllocationDecider doesn't realize I'm shutting down all my warm nodes and IMHO it should.

/cc @dakrone @bczifra 
x-pack/plugin/autoscaling/src/main/java/org/elasticsearch/xpack/autoscaling/storage/ReactiveStorageDeciderService.java
allocation.debugDecision(false);
            }
        }
        private boolean canAllocate(ShardRouting shard, RoutingAllocation allocation) {
            return nodesInTier(allocation.routingNodes()).anyMatch(
                node -> allocationDeciders.canAllocate(shard, node, allocation) != Decision.NO
            );
        }
        boolean needsThisTier(ShardRouting shard, RoutingAllocation allocation) {
            if (isAssignedToTier(shard, allocation) == false) {
                return false;
            }
            IndexMetadata indexMetadata = indexMetadata(shard, allocation);
            Set<Decision.Type> decisionTypes = allocation.routingNodes()
                .stream()
                .map(node -> DataTierAllocationDecider.shouldFilter(indexMetadata, node.node(), this::highestPreferenceTier, allocation))
                .map(Decision::type)
                .collect(Collectors.toSet());
            if (decisionTypes.contains(Decision.Type.NO)) {
                
                return decisionTypes.size() == 1;
            }
            
            if (shard.primary() == false) {
                return false;
            }
            assert allocation.debugDecision() == false;
            
            allocation.debugDecision(true);
            try {