### Elasticsearch Version

many

### Installed Plugins

none

### Java Version

_bundled_

### OS Version

all

### Problem Description

The `filter_by_filter` optimization should kick in when it makes aggs faster. We have a switch to turn it off, but we advise folks not to because it usually does a good job. Well, we've found a case where the heuristics do a bad job of picking the implementation - specifically I've seen a case when the `_doc_count` field exists and we're slower. I'm not really sure if this is universal, but it could be. I'm trying to figure out if this is an all the time thing or a sometimes thing.

### Steps to Reproduce

index docs with `_doc_count`. run a terms agg on a field with a cardinality in the ball park of 500.

### Logs (if relevant)

_No response_
server/src/main/java/org/elasticsearch/search/aggregations/bucket/filter/FilterByFilterAggregator.java
* field.
     */
    private int segmentsWithDocCountField;
    /**
     * Count of segments this aggregator performed a document by document
     * collection for. We have to collect when there are sub-aggregations
     * and it disables some optimizations we can make while just counting.
     */
    private int segmentsCollected;
    /**
     * Count of segments this aggregator counted. We can count when there
     * aren't any sub-aggregators and we have some counting optimizations
     * that don't apply to document by document collections.
     * <p>
     * But the "fallback" for counting when we don't have a fancy optimization
     * is to perform document by document collection and increment a counter
     * on each document. This fallback does not increment the
     * {@link #segmentsCollected} counter and <strong>does</strong> increment
     * the {@link #segmentsCounted} counter because those counters are to
     * signal which operation we were allowed to perform. The filters
     * themselves will have debugging counters measuring if they could
     * perform the count from metadata or had to fall back.
     */
    private int segmentsCounted;
    /**
     * Build the aggregation. Private to force callers to go through the
     * {@link AdapterBuilder} which centralizes the logic to decide if this
     * aggregator would be faster than the native implementation.
     */
    private FilterByFilterAggregator(