### Description

![image](https://github.com/elastic/elasticsearch/assets/76245/d0443170-0419-4759-b505-7620ff3d6594)

The execution time of grouping stats by multiple keywords has increased: 1.2 to 1.6 seconds (with doc_partitioning), 4.3 to 6.7 seconds (with shard_partitioning). The equivalent query with the _search API takes 1.8 seconds.

`from nyc_taxis | ... | stats c = count(dropoff_datetime) by trip_type, payment_type, vendor_id`

This regression occurred since July 2, 2023, and I suspect it may be due to the PR: [Support multivalued fields on arbitrary grouping](https://github.com/elastic/elasticsearch-internal/pull/1340).

/cc @nik9000
x-pack/plugin/esql/compute/src/test/java/org/elasticsearch/compute/operator/MultivalueDedupeTests.java
Set<BytesRef> previousValues = new HashSet<>(prevSize);
                while (previousValues.size() < prevSize) {
                    previousValues.add(new BytesRef(randomAlphaOfLengthBetween(1, 20)));
                }
                assertBytesRefHash(previousValues, b);
            }
            case INT -> {
                int prevSize = between(1, 10000);
                Set<Integer> previousValues = new HashSet<>(prevSize);
                while (previousValues.size() < prevSize) {
                    previousValues.add(randomInt());
                }
                assertIntHash(previousValues, b);
            }
            case LONG -> {
                int prevSize = between(1, 10000);
                Set<Long> previousValues = new HashSet<>(prevSize);
                while (previousValues.size() < prevSize) {
                    previousValues.add(randomLong());
                }
                assertLongHash(previousValues, b);
            }
            case DOUBLE -> {
                int prevSize = between(1, 10000);
                Set<Double> previousValues = new HashSet<>(prevSize);
                while (previousValues.size() < prevSize) {
                    previousValues.add(randomDouble());
                }
                assertDoubleHash(previousValues, b);
            }