Fire - Notice history

Beta Bot experiencing major outage

Bot - Operational

100.0% uptime
Feb 2023 · 100.0%Mar · 100.0%Apr · 100.0%

Previous page

Next page

Feb 2023
Mar 2023
Apr 2023

Beta Bot - Major outage

Website - Operational

100.0% uptime
Feb 2023 · 100.0%Mar · 100.0%Apr · 100.0%

Previous page

Next page

Feb 2023
Mar 2023
Apr 2023

Notice history

View current status

Apr 2023

Invite related features not working
ResolvedDegraded performance66 hours 27 minutes
  • Resolved
    UTC
    Resolved

    This incident has been resolved.

  • Update
    UTC
    Update

    I've made some temporary changes to allow some services to continue operating and have turned off some internal services that use invites to try and prevent further issues It's unknown if these changes will work for the entire duration of the ratelimit or not as they are very temporary and not as reliable The initial cause is still unknown but I have been able to rule out the bot as the issue

  • Investigating
    UTC
    Investigating

    It seems Fire has gotten caught in a day long ratelimit from fetching invites. As of now, I'm not sure what exactly caused it but until the ratelimit expires, features that rely on fetching invites from Discord will not work. I will update this post if I find out what has caused this and will try to see if I can do anything to prevent it in the future

Mar 2023

Fire is offline in some servers
ResolvedPartial outage3 hours 25 minutes
  • Resolved
    UTC
    Resolved

    This incident has been resolved.

  • Monitoring
    UTC
    Monitoring

    An update has been deployed that should reduce the impact of this issue (or at least delay it enough to find the source) if it were to occur again. The source ended up being a server that is abusing Discord's new Clyde AI feature to make 200,000+ threads, resulting in this cluster hitting the memory limit and restarting I am now monitoring everything to ensure it's all working smoothly

  • Identified
    UTC
    Identified

    The issue has been identified but unfortunately to discover the source, all clusters may be impacted temporarily. You may see this impact as the bot going offline for brief periods of time every so often

  • Investigating
    UTC
    Investigating

    Unfortunately, one of Fire's clusters is having issues staying online which results in the bot being offline for some servers. I am working as hard as I can to figure out the issue and get it back online completely but this is proving quite difficult as the issue disappears as soon as it disconnects from Discord or if it gets auto-restarted

Services Unavailable
Resolved42 minutes
  • Resolved
    UTC
    Resolved

    This incident has been resolved.

  • Investigating
    UTC
    Investigating

    In an attempt to resolve issues with Fire's deploy system, I restarted the VPS it runs on and now PM2 is refusing to work meaning none of the applications that were running are coming back online. I am working on restoring everything and will try to do so as quickly as I can (especially since the Oscars are in just over an hour)

Feb 2023

No notices reported this month

Previous

Feb 2023 to Apr 2023

Next