### Elasticsearch Version

many

### Installed Plugins

none

### Java Version

_bundled_

### OS Version

all

### Problem Description

The `filter_by_filter` optimization should kick in when it makes aggs faster. We have a switch to turn it off, but we advise folks not to because it usually does a good job. Well, we've found a case where the heuristics do a bad job of picking the implementation - specifically I've seen a case when the `_doc_count` field exists and we're slower. I'm not really sure if this is universal, but it could be. I'm trying to figure out if this is an all the time thing or a sometimes thing.

### Steps to Reproduce

index docs with `_doc_count`. run a terms agg on a field with a cardinality in the ball park of 500.

### Logs (if relevant)

_No response_
server/src/main/java/org/elasticsearch/search/aggregations/support/AggregationContext.java
*/
    public final FieldContext buildFieldContext(String field) {
        MappedFieldType ft = getFieldType(field);
        if (ft == null) {
            
            return null;
        }
        return new FieldContext(field, buildFieldData(ft), ft);
    }
    /**
     * Returns an existing registered analyzer that should NOT be closed when finished being used.
     * @param analyzer The custom analyzer name
     * @return The existing named analyzer.
     */
    public abstract Analyzer getNamedAnalyzer(String analyzer) throws IOException;
    /**
     * Creates a new custom analyzer that should be closed when finished being used.
     * @param indexSettings The current index settings or null
     * @param normalizer Is a normalizer
     * @param tokenizer The tokenizer name or definition to use
     * @param charFilters The char filter name or definition to use
     * @param tokenFilters The token filter name or definition to use
     * @return A new custom analyzer
     */
    public abstract Analyzer buildCustomAnalyzer(
        IndexSettings indexSettings,
        boolean normalizer,
        NameOrDefinition tokenizer,
        List<NameOrDefinition> charFilters,
        List<NameOrDefinition> tokenFilters