Network Status

[resolved]

[DE, FRA-10] Connectivity issues in part of our network [Update]

Dear LeaseWeb Customer, 

We are currently experiencing connectivity issues in part of our network in our FRA-10 datacenter

We are investigating the cause of these issues. We will update you once we have more information.

The following network prefixes might be affected: 

37.58.54.0/24 
37.58.55.0/24 
37.58.61.0/24 
37.58.62.0/24
37.58.63.0/24 
46.165.218.176/28 
46.165.219.0/24 
46.165.236.0/24 
46.165.237.32/28
46.165.238.0/24 
46.165.252.0/23
78.159.126.0/25 
84.16.224.0/20
178.162.171.128/25
178.162.192.0/19
185.17.144.0/22
212.95.36.0/24 
2A00:C98:2050::/44

77.223.144.0/23
185.17.32.0/22
185.28.69.0/24
212.7.202.0/26
212.7.202.64/27
185.151.204.0/22

[UPDATE 04:05 CEST]

Due to a cooling issue in the FRA-10 datacenter, our routers are overheated and fabric modules are failing. The datacenter engineers are working on fixing the cooling. We will keep you updated.

[UPDATE 04:25 CEST]

We were forced to power down the overheated routers. We are waiting for the temperature to go down in the datacenter before powering them back up. We will keep you updated.

[UPDATE 05:08 CEST]

Since 05:00 CEST all DE services are unreachable. Due to the temperature not going down we were forced to power down the core routers. The datacenter engineers are trying to fix the problem. We will update you as soon as we have more information.

[Update 06:35 CEST]
Engineers have powered down racks to decrease load on cooling system. We enabled power to one of our core routers and it is back online. Network connectivity has been restored for a portion of our customer. Some of our routers are still powered down because of the cooling issue. We will update you as soon as more information is available.

[Update 07:20 CEST]
Still working hard on the resolution. The cooling issue is found and the Cooling Specialist is working on the fix. 
 We will update you as soon as more information is available.

[Update 08:00 CEST]
The cooling system is running again. As soon we have reached a good temperature, we will power up the routers and servers again.

[Update 08:25 CEST]
We are powering on all affected routers as a priority over other equipment.

[Update 08:35 CEST]
All routers are back online.

[Update 09:30 CEST]
Routers and switches are up and running.
The datacenter room has cooled enough to power on the remaining servers.

[Update 10:30]
The issue has been resolved. Servers should be back online within the next minutes. If you still experience issues, please contact our support via support@de.leaseweb.com or open a ticket via the Customer Portal. 
Our apologies for the caused inconvenience. 
We are in contact with our vendor in order to take the appropriate actions and prevent the issue from occurring again.

Sincerely, LeaseWeb http://www.leasewebnoc.com Data center naming system is explained here: http://www.leaseweb.com/en/about-us/data-centers

Below, you will find the network status for the LeaseWeb network. If you would like to be notified about network status changes and maintenance windows, you can subscribe to the LeaseWeb NOC mailinglist.