### Elasticsearch Version

many

### Installed Plugins

none

### Java Version

_bundled_

### OS Version

all

### Problem Description

The `filter_by_filter` optimization should kick in when it makes aggs faster. We have a switch to turn it off, but we advise folks not to because it usually does a good job. Well, we've found a case where the heuristics do a bad job of picking the implementation - specifically I've seen a case when the `_doc_count` field exists and we're slower. I'm not really sure if this is universal, but it could be. I'm trying to figure out if this is an all the time thing or a sometimes thing.

### Steps to Reproduce

index docs with `_doc_count`. run a terms agg on a field with a cardinality in the ball park of 500.

### Logs (if relevant)

_No response_
server/src/test/java/org/elasticsearch/search/aggregations/bucket/filter/FiltersAggregatorTests.java
FiltersAggregationBuilder builder = new FiltersAggregationBuilder(
            "t",
            
            new KeyedFilter("k", new RangeQueryBuilder("test").from("2020-01-01").to("2020-02-01"))
        );
        withAggregator(builder, topLevelQuery, iw -> {
            /*
             * There has to be a document inside the query and one outside
             * the query or we'll end up with MatchAll or MathNone.
             */
            long time = DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER.parseMillis("2010-01-02");
            iw.addDocument(List.of(new LongPoint("test", time), new SortedNumericDocValuesField("test", time)));
            time = DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER.parseMillis("2020-01-02");
            iw.addDocument(List.of(new LongPoint("test", time), new SortedNumericDocValuesField("test", time)));
            time = DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER.parseMillis("2020-01-01");
            iw.addDocument(List.of(new LongPoint("test", time), new SortedNumericDocValuesField("test", time)));
            time = DateFieldMapper.DEFAULT_DATE_TIME_FORMATTER.parseMillis("2020-02-01");
            iw.addDocument(List.of(new LongPoint("test", time), new SortedNumericDocValuesField("test", time)));
        }, (searcher, aggregator) -> {
            /*
             * The topLevelQuery is entirely contained within the filter query so
             * it is good enough to match that. See MergedPointRangeQueryTests for
             * tons more tests around this. Really in this test we're just excited
             * to prove that we unwrapped the IndexOrDocValuesQuery that the date
             * field mapper adds
             */
            QueryToFilterAdapter<?> filter = ((FiltersAggregator) aggregator).filters().get(0);
            assertThat(filter.query(), equalTo(((IndexOrDocValuesQuery) topLevelQuery).getIndexQuery()));
            Map<String, Object> debug = new HashMap<>();
            filter.collectDebugInfo(debug::put);