**Apache Airflow version**: 2.1.0

**Browsers**: Chrome and Firefox

**What happened**:

Before upgrading to 2.1.0

![before](https://user-images.githubusercontent.com/14293802/120359517-c1ca1100-c2d5-11eb-95ba-58ccc0a3ac37.png)

After upgrading to 2.1.0

![after](https://user-images.githubusercontent.com/14293802/120359528-c4c50180-c2d5-11eb-8e04-f34846ea2736.png)

**What you expected to happen**:

Show/Edit/Delete under Security -> Users are available

<!-- What do you think went wrong? -->

**How to reproduce it**:

Go to Security -> Users (as an admin of course) 
airflow/www/views.py
"""Update edges_to_add and edges_to_skip according to TaskGroups."""
        if isinstance(task_group, BaseOperator):
            return
        for target_id in task_group.downstream_group_ids:
            
            
            target_group = task_group_map[target_id]
            edges_to_add.add((task_group.downstream_join_id, target_group.upstream_join_id))
            for child in task_group.get_leaves():
                edges_to_add.add((child.task_id, task_group.downstream_join_id))
                for target in target_group.get_roots():
                    edges_to_skip.add((child.task_id, target.task_id))
                edges_to_skip.add((child.task_id, target_group.upstream_join_id))
            for child in target_group.get_roots():
                edges_to_add.add((target_group.upstream_join_id, child.task_id))
                edges_to_skip.add((task_group.downstream_join_id, child.task_id))
        
        
        
        for target_id in task_group.downstream_task_ids:
            edges_to_add.add((task_group.downstream_join_id, target_id))
            for child in task_group.get_leaves():
                edges_to_add.add((child.task_id, task_group.downstream_join_id))
                edges_to_skip.add((child.task_id, target_id))