Push Notifications are a really cool feature of the modern web platform, but I find they sometimes surprise me. A key part to that is it means you can potentially interact with a site even when the page (or entire browser window) is closed. I have no idea how to temporaily turn off beyond blocking notifications for that site.