Fire - Networking Issues – Incident details

Beta Bot experiencing major outage

Networking Issues

ResolvedMajor outage29 minutes

Affected

Bot

Major outage from 9:08 PM to 9:13 PM, Operational from 9:13 PM to 4:53 PM

Beta Bot

Major outage from 9:08 PM to 9:13 PM, Operational from 9:13 PM to 12:38 AM

Website

Partial outage from 9:08 PM to 9:13 PM, Operational from 9:13 PM to 4:53 PM

Backend

Major outage from 9:08 PM to 9:13 PM, Operational from 9:13 PM to 4:53 PM

Aether

Major outage from 9:08 PM to 9:13 PM, Operational from 9:13 PM to 4:53 PM

  • ResolvedUTC
    This incident has been resolved.
  • MonitoringUTC

    Connectivity appears to have returned and services are now reporting as healthy. Active monitoring will continue for a period of time in the event of issues returning

  • InvestigatingUTC

    It seems Fire and related services are currently unavailable due to possible networking issues. Services are running but it seems that all connectivity on both machines (production & staging) has been lost

    It's unknown what exactly is causing this issue but my guess is Hetzner due to how both machines lost connectivity at the same time and I'm unable to access the Hetzner console