The fact that it doesn't leave your browser is not something that you can rely on. If they get compromised, it doesn't matter whether it goes to the server or not - the same code that keeps it in the browser today comes from that server.

In event of a compromise, it could just serve you code that uploads your keys. How it works today is irrelevant to the security model.