Fire - Notice history

Due to a misplaced curly brace in Fire's message update listener, some actions weren't being triggered. A fix has been deployed for the issue and it should no longer occur. Apologies for the inconvenience.

During this time, I will be restarting Fire to deploy bug fixes while also doubling the cluster count to 4 and having a shard per cluster. This will hopefully reduce delays during reboots and will allow me to have finer control over which servers get an update first. Fire will be offline for a couple minutes while I make these changes. I will eventually have a way to change the cluster & shard counts without complete downtime but did not find it was necessary as of yet.

Fire's VPS has been having issues with networking over the past day and a half. I didn't open an incident yesterday since I assumed it was just going to be temporary and it was out of my control anyways but since the issues are still ongoing, I decided I should open an incident to let people know what's happening. I am still unsure on the root cause of the issues and sincerely apologise for the inconvenience.

Discord is returning 503s on all of Fire's requests at the moment meaning the bot is currently unable to perform any actions.

Discord is currently returning 503 for some of Fire's requests (and most requests in my client) You may not receive responses from Fire when attempting to use commands

Fire is currently seeing an increased number of 500 errors coming from Discord and I can see the same in the client so Discord might be having some issues right now. Reading and sending messages seems to be the most affected so Fire may not be able to respond to commands

It seems Fire's VPS has gotten a temp Cloudflare ban which caused Fire and other services to be offline from at least 3:15 UTC. I have partially recovered Fire & Fire Beta, they are currently running locally for the duration of the ban and will be moved back to the VPS once the ban is lifted (which should be ~4:05 UTC) and I will restore other affected services like Status & inv.wtf too The ban seems to have happened while trying to sync the mute role in a server. Somehow Fire got stuck in a request loop and kept sending requests to update the channel permissions of that server even though they had already been updated resulting in 429s and eventually this ban. It is unknown why Fire got stuck in this loop and I will be investigating tomorrow (as it's currently 4:42 in the morning for me)

Some users and servers may be seeing elevated error rates at the current point in time. This is due to a recent deploy that had a large issue I overlooked. A fix is being worked on and will be implemented soon.

Fire is currently having issues sending requests to certain services, including Discord. This may cause long response times or no response at all. I am currently investigating the cause and attempting to mitigate the issue

I am seeing elevated latency & 5XX responses coming from Discord and duplicated messages. This usually indicates an outage on Discord's end.

It seems the ongoing outage may be affecting Fire as I've noticed some high latency on requests and a few failed interactions. It seems to have calmed down now as it happened a few minutes ago while I was away but I thought I'd open an incident just in case more issues occur

We're experiencing elevated latency on Fire. This is most likely an issue on Discord's end rather than ours. We're currently investigating the issue

Discord seems to be having major issues at the moment and it has started to affect Fire. You can keep up to date with Discord's incidents using the links below: https://stspg.io/mnbt34n1c6nc?u=684bns7mjtnh https://stspg.io/klccjhq8sf47?u=jm3l03t7tr5f