### Preflight Checklist

- [X] I have read the [Contributing Guidelines](https://github.com/electron/electron/blob/master/CONTRIBUTING.md) for this project.
- [X] I agree to follow the [Code of Conduct](https://github.com/electron/electron/blob/master/CODE_OF_CONDUCT.md) that this project adheres to.
- [X] I have searched the [issue tracker](https://www.github.com/electron/electron/issues) for a feature request that matches the one I want to file, without success.

### Electron Version

12.0.7

### What operating system are you using?

Ubuntu

### Operating System Version

5.4.0-73-generic #82~18.04.1-Ubuntu

### What arch are you using?

x64

### Last Known Working Electron version

_No response_

### Expected Behavior

`const existingPermissions = await navigator.serial.getPorts();` should return a list of serial ports for which permission has already been granted, even when a device has been unplugged and plugged back in again.

### Actual Behavior

The list is empty, even when the same code works in the browser (Chromium v90.0.4430.93).

### Testcase Gist URL

https://gist.github.com/gniezen/34950a5fa84dcf000e2b55a8d599ecd6

### Additional Information

To reproduce using gist:

- Replace `usbVendorId` and `usbProductId` with that of your test device.
- Click the `Connect` button. It should show up in the port list on the main console.
- Click the `Get Ports` button. It should show the existing permission in the developer console.
- Unplug and re-plug the device.
- Click the `Get Ports` button. It should show the existing permissions, but returns an empty list.
shell/browser/serial/serial_chooser_context.cc
EnsurePortManagerConnection();
  return port_manager_.get();
}
void SerialChooserContext::AddPortObserver(PortObserver* observer) {
  port_observer_list_.AddObserver(observer);
}
void SerialChooserContext::RemovePortObserver(PortObserver* observer) {
  port_observer_list_.RemoveObserver(observer);
}
base::WeakPtr<SerialChooserContext> SerialChooserContext::AsWeakPtr() {
  return weak_factory_.GetWeakPtr();
}
void SerialChooserContext::OnPortAdded(device::mojom::SerialPortInfoPtr port) {
  for (auto& observer : port_observer_list_)
    observer.OnPortAdded(*port);
}
void SerialChooserContext::OnPortRemoved(
    device::mojom::SerialPortInfoPtr port) {
  for (auto& observer : port_observer_list_)
    observer.OnPortRemoved(*port);
  std::vector<std::pair<url::Origin, url::Origin>> revoked_url_pairs;
  for (auto& map_entry : ephemeral_ports_) {
    std::set<base::UnguessableToken>& ports = map_entry.second;
    if (ports.erase(port->token) > 0) {
      revoked_url_pairs.push_back(map_entry.first);
    }
  }
  port_info_.erase(port->token);
}
void SerialChooserContext::EnsurePortManagerConnection() {