Bot - Operational
Bot
Previous page
Next page
Beta Bot - Operational
Beta Bot
Website - Operational
Website
Expand group
Backend
External
Extras
This incident has been resolved.
In line with Discord's latest update, error rates have returned to normal. I am continuing to monitor error rates and latency.
Discord has (finally) posted an incident on their status page, https://stspg.io/l7cs3n0l1zv2?u=xyby3wl8shc3 This was meant to be posted when they posted the incident but I forgot to hit update, oops!
Fire is currently seeing an increased number of 500 errors coming from Discord and I can see the same in the client so Discord might be having some issues right now. Reading and sending messages seems to be the most affected so Fire may not be able to respond to commands
Fire & Aether are now operational.
Vanity URLs & Redirects are now operational
Services are being brought back online, starting with Status, which should be up and running to send this update.
It seems Fire's VPS has gotten a temp Cloudflare ban which caused Fire and other services to be offline from at least 3:15 UTC. I have partially recovered Fire & Fire Beta, they are currently running locally for the duration of the ban and will be moved back to the VPS once the ban is lifted (which should be ~4:05 UTC) and I will restore other affected services like Status & inv.wtf too The ban seems to have happened while trying to sync the mute role in a server. Somehow Fire got stuck in a request loop and kept sending requests to update the channel permissions of that server even though they had already been updated resulting in 429s and eventually this ban. It is unknown why Fire got stuck in this loop and I will be investigating tomorrow (as it's currently 4:42 in the morning for me)
The fix has been deployed, error rates should be returning to normal (a.k.a non existent since Fire is so good) but if you continue to encounter errors, please join the Fire Discord server (https://inv.wtf/fire) and let me know in the #fire-help channel
Some users and servers may be seeing elevated error rates at the current point in time. This is due to a recent deploy that had a large issue I overlooked. A fix is being worked on and will be implemented soon.
It is still unknown what initially caused the issue but there was have been a brief period of time where Fire's VPS was unable to reach the internet. Fire is once again able to access the internet and should be back in action! I am monitoring network activity to ensure things are back to normal.
Fire is currently having issues sending requests to certain services, including Discord. This may cause long response times or no response at all. I am currently investigating the cause and attempting to mitigate the issue
Latency and error rates have returned to normal, keeping this open for a while so if it happens again I don't need a third incident...
I am seeing elevated latency & 5XX responses coming from Discord and duplicated messages. This usually indicates an outage on Discord's end.
It seems the ongoing outage may be affecting Fire as I've noticed some high latency on requests and a few failed interactions. It seems to have calmed down now as it happened a few minutes ago while I was away but I thought I'd open an incident just in case more issues occur
Error rates have decreased so I have switched one cluster back to the Canary API and will wait to ensure things are working fine before switching the other
Discord's Canary API seems to be having issues so I've switched Fire over to stable so that should eliminate some of the issues. Lowering status to partial outage.
Almost all requests are now 5XX errors so I'm changing the status to major outage.
I've started seeing 502 errors in Fire's logs, changing status to partial outage
We're experiencing elevated latency on Fire. This is most likely an issue on Discord's end rather than ours. We're currently investigating the issue
Discord has marked their incidents as resolved however due to the amount of outages today, I am going to leave this open a little longer just in case.
Discord seems to be having major issues at the moment and it has started to affect Fire. You can keep up to date with Discord's incidents using the links below: https://stspg.io/mnbt34n1c6nc?u=684bns7mjtnh https://stspg.io/klccjhq8sf47?u=jm3l03t7tr5f
Welp, right before I posted the previous update, Discord updated their incident stating the issue is resolved. So I guess this is resolved too now...
From about 1:13 UTC to 1:21 UTC, Fire experienced elevated latency and failed requests. These issues seem to have calmed down for now but it's unknown whether Discord has fully resolved the issue yet. For now, Fire seems to be having minimal issues, so I will change the status to "Degraded Performance" until I have confirmation the issues are resolved
Discord's API is currently having major issues. As I am currently on my phone I am unable to monitor the impact on Fire and other services but I will update this incident with more info as soon as I can
May 2021 to Jul 2021
Next
