I observed a 7.12 cluster that was struggling to complete an election, with at least one node holding over a million `elected-as-master` tasks in its pending task queue. I suspect there were two such nodes because they were just constantly winning elections and then failing because of the next election and so on...

It's currently unclear exactly how we got into this state, but it doesn't seem completely impossible if something else causes a bunch of spurious elections somehow while the master service was blocked. There were some faulty nodes involved for sure.

In any case I believe we can and should take steps to avoid processing stale `elected-as-master` tasks.
server/src/main/java/org/elasticsearch/cluster/coordination/JoinTaskExecutor.java
public JoinTaskExecutor(AllocationService allocationService, RerouteService rerouteService, long term) {
        this.allocationService = allocationService;
        this.rerouteService = rerouteService;
        this.term = term;
    }
    @Override
    public ClusterState execute(ClusterState currentState, List<TaskContext<JoinTask>> joinTaskContexts) throws Exception {
        
        
        
        if (currentState.term() > term) {
            logger.trace("encountered higher term {} than current {}, there is a newer master", currentState.term(), term);
            throw new NotMasterException(
                "Higher term encountered (current: " + currentState.term() + " > used: " + term + "), there is a newer master"
            );
        }
        final boolean isBecomingMaster = joinTaskContexts.stream().anyMatch(t -> t.getTask().isBecomingMaster());
        final DiscoveryNodes currentNodes = currentState.nodes();
        boolean nodesChanged = false;
        ClusterState.Builder newState;
        if (currentNodes.getMasterNode() == null && isBecomingMaster) {
            assert currentState.term() < term : "there should be at most one become master task per election (= by term)";
            
            
            
            newState = becomeMasterAndTrimConflictingNodes(currentState, joinTaskContexts);
            nodesChanged = true;
        } else if (currentNodes.isLocalNodeElectedMaster()) {
            assert currentState.term() == term : "term should be stable for the same master";
            newState = ClusterState.builder(currentState);