Bot - Operational
Bot
Previous page
Next page
Website - Operational
Website
Expand group
Backend
External
Extras
No notices reported this month
This incident has been resolved.
I've made some temporary changes to allow some services to continue operating and have turned off some internal services that use invites to try and prevent further issues It's unknown if these changes will work for the entire duration of the ratelimit or not as they are very temporary and not as reliable The initial cause is still unknown but I have been able to rule out the bot as the issue
It seems Fire has gotten caught in a day long ratelimit from fetching invites. As of now, I'm not sure what exactly caused it but until the ratelimit expires, features that rely on fetching invites from Discord will not work. I will update this post if I find out what has caused this and will try to see if I can do anything to prevent it in the future
An update has been deployed that should reduce the impact of this issue (or at least delay it enough to find the source) if it were to occur again. The source ended up being a server that is abusing Discord's new Clyde AI feature to make 200,000+ threads, resulting in this cluster hitting the memory limit and restarting I am now monitoring everything to ensure it's all working smoothly
The issue has been identified but unfortunately to discover the source, all clusters may be impacted temporarily. You may see this impact as the bot going offline for brief periods of time every so often
Unfortunately, one of Fire's clusters is having issues staying online which results in the bot being offline for some servers. I am working as hard as I can to figure out the issue and get it back online completely but this is proving quite difficult as the issue disappears as soon as it disconnects from Discord or if it gets auto-restarted
In an attempt to resolve issues with Fire's deploy system, I restarted the VPS it runs on and now PM2 is refusing to work meaning none of the applications that were running are coming back online. I am working on restoring everything and will try to do so as quickly as I can (especially since the Oscars are in just over an hour)
Mar 2023 to May 2023
Next