Searching a large number of unavailable shards through e.g. the `*` pattern while a large cluster is recovering from a full restart or so, leads to extremely large responses containing an exception for each shard.


![](https://user-images.githubusercontent.com/6490959/193701098-868888bc-cdc7-4104-a32a-d9c84d74aef2.png)

This example shows a 375M on heap response for ~25k unavailable shards being returned. Over the wire, this serialises with a similar size and we'd have 700M+ peak heap usage for a search request when the valid search response in a green cluster might be much smaller than this.
It seems to me that we could mostly resolve this by not returning the stack trace in this specific case of the shard not available exception (it doesn't seem valuable for users and we made a similar fix around unavailable shards in snapshots state responses)? 
server/src/main/java/org/elasticsearch/action/search/AbstractSearchAsyncAction.java
* referred to as the {@code shardIndex}.
 * The fan out and collect algorithm is traditionally used as the initial phase which can either be a query execution or collection of
 * distributed frequencies
 */
abstract class AbstractSearchAsyncAction<Result extends SearchPhaseResult> extends SearchPhase implements SearchPhaseContext {
    private static final float DEFAULT_INDEX_BOOST = 1.0f;
    private final Logger logger;
    private final SearchTransportService searchTransportService;
    private final Executor executor;
    private final ActionListener<SearchResponse> listener;
    private final SearchRequest request;
    /**
     * Used by subclasses to resolve node ids to DiscoveryNodes.
     **/
    private final BiFunction<String, String, Transport.Connection> nodeIdToConnection;
    private final SearchTask task;
    protected final SearchPhaseResults<Result> results;
    private final ClusterState clusterState;
    private final Map<String, AliasFilter> aliasFilter;
    private final Map<String, Float> concreteIndexBoosts;
    private final SetOnce<AtomicArray<ShardSearchFailure>> shardFailures = new SetOnce<>();
    private final Object shardFailuresMutex = new Object();
    private final AtomicBoolean hasShardResponse = new AtomicBoolean(false);
    private final AtomicInteger successfulOps = new AtomicInteger();
    private final AtomicInteger skippedOps = new AtomicInteger();
    private final SearchTimeProvider timeProvider;
    private final SearchResponse.Clusters clusters;
    protected final GroupShardsIterator<SearchShardIterator> toSkipShardsIts;
    protected final GroupShardsIterator<SearchShardIterator> shardsIts;
    private final SearchShardIterator[] shardIterators;