When #65905 was implemented to automatically set the heap size the idea was that explicit settings provided by the user would be respected:

> In all cases we short-circuit if a user provides explicit heap options
> so we only ever auto-determine heap if no existing heap options exist.

This doesn't always happen.  In particular, it only happens if the user provided setting is an exact multiple of 4MB.  Otherwise, the JVM rounds up what the user specified to the next highest multiple of 4MB and the setting is not then considered entirely a command line setting.  For example:

```
bash-3.2$ java -Xmx408m -XX:+PrintFlagsFinal | grep MaxHeapSize
   size_t MaxHeapSize                              = 427819008                                 {product} {command line}
bash-3.2$ java -Xmx409m -XX:+PrintFlagsFinal | grep MaxHeapSize
   size_t MaxHeapSize                              = 429916160                                 {product} {command line, ergonomic}
```

So, on my laptop if I start Elasticsearch with `-Xmx408m` I get a 408MB heap.  If I start Elasticsearch with `-Xmx409m` I get a 31GB heap.  (If the `-Xmx409m` wasn't replaced then I'd get a 412MB heap, i.e. 409 rounded up to the next multiple of 4.)

I doubt this is a problem at all for human end users, as they'll generally choose nice round sizes. But the place where it makes a difference is where the `-Xmx` argument for Elasticsearch is provided by a machine orchestrator.  For example, the Elastic Cloud orchestrator used to set the heap size on 1GB ML nodes to 409MB prior to 8.2.  We thought that was taking precedence over the Elasticsearch auto-sizing but actually these 1GB ML nodes have been using Elasticsearch's auto-sizing since 7.11.  It doesn't matter since the calculations were deliberately designed to be the same to start off with.

If we want to make the auto-sized heap respect all user overrides then I think this line should be changed to check if `origin` contains `command line` instead of being equal to it: https://github.com/elastic/elasticsearch/blob/62514c55a0674a626b142602bdf27eb1b189bcd5/distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/JvmOption.java#L45
distribution/tools/server-cli/src/main/java/org/elasticsearch/server/cli/JvmOption.java
public static boolean isMinHeapSpecified(final Map<String, JvmOption> finalJvmOptions) {
        JvmOption minHeapSize = finalJvmOptions.get("MinHeapSize");
        return minHeapSize != null && minHeapSize.isCommandLineOrigin();
    }
    public static boolean isInitialHeapSpecified(final Map<String, JvmOption> finalJvmOptions) {
        JvmOption initialHeapSize = finalJvmOptions.get("InitialHeapSize");
        return initialHeapSize != null && initialHeapSize.isCommandLineOrigin();
    }
    public static long extractMaxDirectMemorySize(final Map<String, JvmOption> finalJvmOptions) {
        return Long.parseLong(finalJvmOptions.get("MaxDirectMemorySize").getMandatoryValue());
    }
    /**
     * Determine the options present when invoking a JVM with the given user defined options.
     */
    public static Map<String, JvmOption> findFinalOptions(final List<String> userDefinedJvmOptions) throws InterruptedException,
        IOException {
        return flagsFinal(userDefinedJvmOptions).stream()
            .map(OPTION::matcher)
            .filter(Matcher::matches)
            .collect(Collectors.toUnmodifiableMap(m -> m.group("flag"), m -> new JvmOption(m.group("value"), m.group("origin"))));
    }
    private static List<String> flagsFinal(final List<String> userDefinedJvmOptions) throws InterruptedException, IOException {
        /*
         * To deduce the final set of JVM options that Elasticsearch is going to start with, we start a separate Java process with the JVM
         * options that we would pass on the command line. For this Java process we will add two additional flags, -XX:+PrintFlagsFinal and
         * -version. This causes the Java process that we start to parse the JVM options into their final values, display them on standard
         * output, print the version to standard error, and then exit. The JVM itself never bootstraps, and therefore this process is
         * lightweight. By doing this, we get the JVM options parsed exactly as the JVM that we are going to execute would parse them
         * without having to implement our own JVM option parsing logic.
         */