10:35 – Packet loss identified.
10:50 – After further investigation isolated the issue to our peering1 router in London.
11:05 – Traffic re-routed around the peering1 router.
11:15 – Further investigation concludes faulty card in peering router. Maintenance work is scheduled to replace the faulty network interface on the Juniper router.

As a valued customer, we thank you for your understanding during this period.

16:48 – Power to site lost – running on UPS.
17:15 – UPS systems depleted, generators failed to start.
17:30 – Generators failovers failed to function despite multiple attempts – The power engineers were dispatched.
18:32 – The power engineers arrived on site.
18:54 – The power engineer estimates 30-45 minutes to return power to site.
19:16 – The power was returned to site and the process of booting up each rack commenced.
19:35 – All racks powered up and brought on-line.

The last twenty-four hours have seen numerous power grid blips but the system as designed have taken the load with the UPS battery backup and generators keeping the facility live, ensuring an un-interrupted service.

Tonight however, at 16:48 a failure in the control boards prevented the generators from powering the facility once the power from the grid failed, a manual bypass was installed to get the facility on-line.
The facility power engineers are continuing to monitor the facility and extra staff were drafted in to help.

If you are still experiencing any technical issues please contact the support desk via https://support.coreix.net/, however if you have any further questions or would like any clarification please do hesitate to contact the Operations Director, Shazad Boota directly at s.boota@coreix.net

As a valued customer, we thank you for your understanding during this period.