Fire - Moderation/Action/Member logs not being sent – Incident details

This incident has been resolved.

Fix has been deployed to both clusters and the log queue should've been recovered for guilds on Cluster 1 though it may have needed to attempt another log to actually send the queue. I'm monitoring logging in one of the larger guilds Fire is in to ensure there's no lingering issues.

After deploying the fix and seeing it not work, I noticed I had forgot to add some changes to the commit so it was only half fixed. Full fix being pushed and deployed now 😅

The fix has been pushed and is ready to deploy. Due to logging being broken on Cluster 0 after the failed fix, I will not be able to recover items currently in the queue. As for Cluster 1, I should be able to recover the queue once the fix has deployed to Cluster 0

The issue has been identified and a fix is being worked on

Seems the fix just broke it even more, back to the drawing board I go...

A potential fix has been made and pushed. I will now attempt to send any messages currently in the queue for all guilds on Cluster 0 and then deploy the update.