On August 30, Azure’s Australia East data center had a big problem, affecting customers like Bank of Queensland and Jetstar. Here’s the timeline:
30 August 2023 @ 08:41 – Voltage sag occurred on utility power line 30 August 2023 @ 08:43 – Five chillers failed to restart 30 August 2023 @ 10:30 – Storage and SQL alerted by monitors about failure rates 30 August 2023 @ 10:57 – Cosmos DB Initial impact detected via monitoring 30 August 2023 @ 11:15 – Attempts to stabilize the five chillers were unsuccessful after multiple chiller restarts 30 August 2023 @ 11:34 – Decision