Affecting Other - Complete Data Center Relocation
Maintenance beginning Friday October 11th 2024 at 6:00PM EST / 3:00PM PST in our Phoenix Arizona Data Center (AZ1) ONLY.
We will be relocating our existing AZ1 (Phoenix AZ) rack space into a different facility in the same region.
We expect approximately 2-3 hours of downtime during the maintenance process for the relocation, including drive time.
Maintenance is expected to be reasonably quick and occur smoothly due to advance preparation by the new facility/team.
However, it takes time to de-rack equipment, drive to the new facility (just 20 minutes away), and re-rack equipment.
IP addresses will NOT need to change, though there may be a short period of time while routes update across the Internet.
Affecting Other - Complete Data Center Relocation
Maintenance beginning Wednesday May 29th 2024 at 9:00PM PST / 6:00PM EST in our Edison New Jersey Data Center (NJ2) ONLY.
We will be relocating/collapsing our existing NJ2 rack space into the nearby NJ1 facility in the same region.
We have been informed to expect 4-6 hours of downtime during the maintenance window for the relocation.
We apologize for the required outage, this was necessary due to expiring contracts and excessive price increases at the NJ2 facility.
Maintenance is expected to be reasonably quick and occur smoothly due to advance preparation by the NJ1 facility/team.
Affecting Other - Power Maintenance
Notification for our Portland OR Data Center (OR1):
The data center is upgrading the power infrastructure in the suite at the OR data center, and a power shutdown will be needed for approximately one hour, starting at 11:50PM PST on Friday, February 1st 2024, through 1:00AM PST.
Staff will be standing by in shifts during and after the window to ensure that all equipment is powered back on. Thank you for your patience and understanding. We acknowledge that planned power shutdowns can be highly disruptive and that these should occur only sparingly, if at all. While the upgrade requires this outage window, it will allow for enhanced power redundancy and growth at this location for the future.
Affecting Other - Complete Data Center Outage
Chicago IL Data Center (IL1) - Outage - Entire location outage.
Cooling system failure, forcing data center shutdown. Currently waiting for remedy by data center facility operators.
Affecting Other - Complete Data Center Outage
Wisconsin Data Center (WI1) - Outage - Entire location outage.
Currently waiting for remedy by data center operators.
UPDATE:
Network issue resolved. Dedicated Server services all restored. VPS node sunder going updates/patching, and will be back online very shortly tonight.
Affecting System - Virtualization Platform
Outage / High Loads - Virtualization (VPS) Platform. Node Specific.
Many VPS nodes down for base O/S re-installation to clean up from prior DDOS/hacker issues and secure systems. Service will be restored per node as each process completed.
UPDATE 1:
We are currently working to restore service to nodes affected from prior ddos/hack issues.
While working smoothly one server at a time to clean up the prior issue, the hacker got angry and decided to corrupt the O/S on all systems he/she/they still could get to that were not re-installed/patched yet.
We are now scrambling to re-install all those nodes, and have been at it non-stop since last night.
The good news is, there has been no breach to any re-install nodes with the most recent version/patch we have been using for weeks now, so it is just a matter of working thru the platform to re-install/patch each and every node.
We do not have an ETA for full resolution yet, but all services should be restored today at some point, with nodes coming on line one by one as completed.
UPDATE 2:
Most nodes should be back online and updated/patched and all VPS back in service. Please contact us via ticket if you have a VPS still down as of this time.
There may be further maintenance on individual nodes not updated/patched yet, brief one-hour on each, but not the ones that were already down from this issue up until now.
UPDATE 3:
Hardware nodes re-installed and patched, so issue should be resolved, as long as hacker doesn't find a new breach.
Client VPS images/data was safe/untouched, this only affected the underlying node systems.
If you still having an issue, please respond back to this ticket, and we will investigate the individual server specifically.
Affecting System - Los Angeles CA VPS Cluster
The Las Vegas CA site is experiencing an issue with the ProxMox clustering system, the management backend that controls the virtualization platform.
The issue is being actively worked on, and should be resolved shortly.
We will be migrating off the legacy ProxMox platform as soon as possible, as it has proven to be unreliable and non-enterprise with it's poor design.
We are just waiting on another software vendor to complete the migration scripts to do so.
Affecting System - Los Angeles CA VPS Cluster
The Las Vegas CA site is experiencing a major issue with the ProxMox clustering system, the management backend that controls the virtualization platform.
We have no ETA on resolution, but it is actively being worked on until resolved.
We will be migrating off the legacy ProxMox platform as soon as possible, as it has proven to be unreliable and non-enterprise with it's poor design.
We are just waiting on another vendor to complete the migration scripts to do so, then we will migrate all ProxMox based VPS on to our enterprise-level proven platform already in production.
Affecting Other - Los Angeles CA Datacenter
Los Angeles CA Datacenter Maintenance will occur on Wednesday afternoon/evening, November 10th 2021, starting at 3PM PST thru approximately 11:59PM PST.
There will be rolling outages, as we relocate each server/node into a larger rack space, to allow for future growth.
Each individual server/node is expected to be about 20-30 minutes down per node, with 1-2 servers being relocated at a time. Each VPS will only be down during individual server move, not during whole maintenance window.
Affecting System - Few servers will be effected
Dallas Data Center schedule to maintenance scheduled Wednesday November 10th at 8PM PST.
Few servers will be effected with downtime in the process.
Dallas Data Center schedule to maintenance tomorrow, Saturday morning - 10-11am CST.
All Dallas servers will be effected with downtime in the process.
Hello,
We having issue with c06 that been reported. we are working to restored the services.
Update 1 : The issue has been fixed. Please if you still have issue submit ticket.
Affecting Server - c15
We have issue with node c15 requiring quick rebooting.
Update : Server have connection crashing we are working into it.
Update 2 : Issue resolved.
Affecting System - NJ Servers
Greetings!
Update : All services online.
Update : Waiting fire up our network routed at new network.
Update : All servers has been moved We are waiting for network engineer to backup our prefixes.
On May 22, 2021 (9.00 AM EST to 9:00 PM EST) we will be migrating all of our hardware from our provider in New Jersey to our new datacenter with Leaseweb. Our location still remains in New Jersey on the East Coast. we will continually update throughout the migration process. As we are physically moving hardware there will be some downtime associated with the migration. We expect downtime to last no more than 12 hours, however this is subject to change.
We appreciate your patience and understanding while we perform the migration to our new data center! Please do not hesitate to reach out to our support team if you have any questions.
Thank you!
Affecting System - Network Connection
We are reaching out to inform you about an upcoming scheduled network
maintenance to the core router which will cause service interruption . The major scheduled maintenance network is to increase a
big part of our network in New Jersey.
Details -
Date: Monday, March 8th, 2021 / Tuesday, March 9th
Start Time: 6:00AM (GMT-6)
Work Window: Window : 6:00AM --11:00AM
Outage Estimated Duration: 30mins to 1 hour
Effect: Major Scheduled Outage
Facilities: New Jersey, East Coast
If you have any questions about this maintenance or want more information on how
it will relate to your specific services with us, please feel free to reply to
this email.
Affecting System - Dallas, TX
We are reaching out to inform you about an upcoming scheduled network
maintenance to the core router which will cause service interruption . The major scheduled maintenance network is to increase a
big part of our network in Dallas.
Details -
Date: Tuesday, February 4th, 2021
Start Time: 6:00AM (GMT-6)
Work Window: Window : 6:00AM -- 8:00AM
Outage Estimated Duration: 30mins to 1 hour
Effect: Major Scheduled Outage
Facilities: Dallas, TX
If you have any questions about this maintenance or want more information on how
it will relate to your specific services with us, please feel free to reply to
this email.
C04 Cluster Los Angeles scheduled Maintenance.
Details -
Work Start Time: Saturday Jan 30, 2020 10:00 AM PDT / 8:00 PM GMT +2
Work End Time: Saturday Jan 30, 2020 10:20 AM PDT / 8:20 PM GMT +2
Work Window: 1 to 2 Hours
Actual Outage Estimated Time: 20 minutes
Effect: C04 Cluster Los Angeles Location
Thanks
Affecting Server - c14
C14 Cluster Los Angeles scheduled Maintenance.
Details -
Work Start Time: Sunday Jan 17, 2020 10:00 AM PDT / 8:00 PM GMT +2
Work End Time: Sunday Jan 17, 2020 10:20 AM PDT / 8:20 PM GMT +2
Work Window: 1 to 2 Hours
Actual Outage Estimated Time: 20 minutes
Effect: C14 Cluster Los Angeles Location
Thanks
Affecting Other - Network Hardware upgrade
|
|
START TIME: |
01/21/2021 10:00 PM EST |
|
|
|
|
END TIME: |
01/22/2021 02:00 AM EST |
|
|
|
|
WHAT: |
Hardware Upgrade |
|
|
|
|
IMPACT: |
None Expected |
||
network status to received all network updates and Maintenance
https://my.letbox.com/public-status.php?statid=1012cf5f68f40b43670a2694454f802a
Affecting System - Loa Angeles Network
Description:
The Why -
We have seen an increase of high packet-per-second attacks compared to volumetric that cause degradation throughout the infrastructure. The upgrade will resolve specific areas that were exploited during these high packet-per-second application attacks. We are also taking the time to perform dark fiber maintenance by upgrading the capacity between all our locations as we have seen our traffic throughout triple within the year.
Though the work window may be long, the actual estimated outage time is only 10-15 minutes, but can happen at any time within the detailed window below:
Details -
Work Start Time: September 10, 2016 2:00 PM PDT
Work End Time: September 10, 2016 6:00 PM PDT.
Work Window: 4 Hours
Actual Outage Estimated Time: 10-15 minutes or less
Effect: Network Outage
Facilities: 624 Wilshire Blvd (1 Wilshire)
— Affecting Connectivity in 700 Wilshire Blvd (LAX2) & 609 W. 7th St. (LAX1), All Los Angeles, CA Clients
The long maintenance window will allow our engineers to roll back in the event of network degradation or outages. We’ve been preparing for this upgrade/maintenance for the last few weeks ensure things go as smoothly as possible and with as little downtime possible.
If you have any questions on the process or any doubts please feel free to contact us anytime!
Thank you,
Affecting System - Los Angeles
Description:
The focus of this maintenance will be to upgrade our Los Angeles Juniper core router’s operating systems which will require rebooting and cause a brief network outage. We’ve run into several bugs in Junos OS (Juniper router operating system) which has hindered network performance recently, so for this reason we can no longer delay the upgrade to the latest stable version of the OS.
Details -
Date: Sunday, June 19, 2016
Time: 2PM PDT
Work Window: 3-5 Hours beginning at 2PM PDT
Actual Outage Estimated Time: 10 minute or less
Effect: Network Outage
Facilities: LAX1 & LAX2
If you have any questions on the process or any doubts please feel free to contact us anytime!
Affecting System - Cisco IOS
Los Angeles & Dallas. The focus of this maintenance will be to upgrade our network distribution routers to the latest versions of Cisco IOS. Though no outage exceeding 1 minute is expected, worst case scenario would be a 10 minute outage as the router reboots and loads the new version of the OS.
Details -
Date: Saturday October 10, 2015
Time: 1AM CST through 5AM CST
Duration: No outage expected, but maintenance window for 4 hours while work is performed.
Effect: Latest versions of IOS for distribution routers
Facilities: Dallas, Texas & Los Angeles, California
If you have any questions on the process or any doubts please feel free to contact us anytime!
Hello,
Affecting System - Dallas Network
DATE: 05/06/2013 12:01 AM CDT
SPAN: 05/06/2013 12:01 AM CDT - 05/06/2013 1:00 AM CDT
DESCRIPTION:
On May 6th 2013 at 12:01 AM CDT CoreXchange engineers will be performing
scheduled maintenance on xxxxx's uplink. Customers connected to
xxxx or customers connected to switches which uplink to
xxxxx will experience an Internet outage. If you are receiving
this notification one or more of your uplinks involve cxswitch51.dllstx03. We
expect the duration of the outage to be no more than five minutes, however
due to the nature of the work it is possible for the outage to last the entire
duration of the maintenance. By 1:00 AM CDT all Internet reachability will be
restored. If you have a redundant uplink there should be no interruption in
service.
Sincerely,
We was suffering from a DDOS Attack with approximately 1.9gbit. The adress has been null routed and you should regain connectivity now.
