There are upcoming maintenance events which may impact our services. Learn more
There are no Open Network Issues Currently
There are 1 Scheduled Network Impacting Issues. Learn more...
IL1 - Chicago IL Data Center - Outage (Resolved) Critical

Affecting Other - Complete Data Center Outage

  • 01/14/2024 15:39 - 01/14/2024 23:30
  • Last Updated 01/14/2024 16:26

Chicago IL Data Center (IL1) - Outage - Entire location outage. 

Cooling system failure, forcing data center shutdown. Currently waiting for remedy by data center facility operators.

 

WI1 - Wisconsin Data Center - Outage (Resolved) Critical

Affecting Other - Complete Data Center Outage

  • 01/05/2024 05:39
  • Last Updated 01/06/2024 18:55

Wisconsin Data Center (WI1) - Outage - Entire location outage. 

Currently waiting for remedy by data center operators.

 

UPDATE: 

Network issue resolved. Dedicated Server services all restored. VPS node sunder going updates/patching, and will be back online very shortly tonight.

VPS Platform Maintenance (Resolved) Low

Affecting System - Virtualization Platform

  • 01/03/2024 23:51 - 01/15/2024 17:11
  • Last Updated 01/06/2024 14:15

Outage / High Loads - Virtualization (VPS) Platform. Node Specific.

Many VPS nodes down for base O/S re-installation to clean up from prior DDOS/hacker issues and secure systems. Service will be restored per node as each process completed.

 

UPDATE 1:

We are currently working to restore service to nodes affected from prior ddos/hack issues.

While working smoothly one server at a time to clean up the prior issue, the hacker got angry and decided to corrupt the O/S on all systems he/she/they still could get to that were not re-installed/patched yet.

We are now scrambling to re-install all those nodes, and have been at it non-stop since last night.

The good news is, there has been no breach to any re-install nodes with the most recent version/patch we have been using for weeks now, so it is just a matter of working thru the platform to re-install/patch each and every node.

We do not have an ETA for full resolution yet, but all services should be restored today at some point, with nodes coming on line one by one as completed.

 

UPDATE 2:

Most nodes should be back online and updated/patched and all VPS back in service. Please contact us via ticket if you have a VPS still down as of this time.

There may be further maintenance on individual nodes not updated/patched yet, brief one-hour on each, but not the ones that were already down from this issue up until now.

 

UPDATE 3:

Hardware nodes re-installed and patched, so issue should be resolved, as long as hacker doesn't find a new breach.

Client VPS images/data was safe/untouched, this only affected the underlying node systems.

If you still having an issue, please respond back to this ticket, and we will investigate the individual server specifically.

 

 

Los Angeles CA Datacenter - Cluster Outage (Resolved) Critical

Affecting System - Los Angeles CA VPS Cluster

  • 02/23/2022 21:47 - 02/24/2022 00:12
  • Last Updated 02/24/2022 00:13

The Las Vegas CA site is experiencing an issue with the ProxMox clustering system, the management backend that controls the virtualization platform.

The issue is being actively worked on, and should be resolved shortly.

 

We will be migrating off the legacy ProxMox platform as soon as possible, as it has proven to be unreliable and non-enterprise with it's poor design.

We are just waiting on another software vendor to complete the migration scripts to do so.

Los Angeles CA Datacenter - Cluster Outage (Resolved) Critical

Affecting System - Los Angeles CA VPS Cluster

  • 02/15/2022 11:00 - 02/15/2022 20:10
  • Last Updated 02/15/2022 17:52

The Las Vegas CA site is experiencing a major issue with the ProxMox clustering system, the management backend that controls the virtualization platform.

We have no ETA on resolution, but it is actively being worked on until resolved.

 

We will be migrating off the legacy ProxMox platform as soon as possible, as it has proven to be unreliable and non-enterprise with it's poor design.

We are just waiting on another vendor to complete the migration scripts to do so, then we will migrate all ProxMox based VPS on to our enterprise-level proven platform already in production.

Los Angeles CA Datacenter Maintenance (Resolved) Critical

Affecting Other - Los Angeles CA Datacenter

  • 11/10/2021 15:00 - 11/10/2021 23:59
  • Last Updated 11/10/2021 23:41

Los Angeles CA Datacenter Maintenance will occur on Wednesday afternoon/evening, November 10th 2021, starting at 3PM PST thru approximately 11:59PM PST.

There will be rolling outages, as we relocate each server/node into a larger rack space, to allow for future growth.

Each individual server/node is expected to be about 20-30 minutes down per node, with 1-2 servers being relocated at a time. Each VPS will only be down during individual server move, not during whole maintenance window. 

LAX maintenance (Resolved) Critical

Affecting System - Few servers will be effected

  • 11/10/2021 20:09 - 11/08/2021 21:03
  • Last Updated 11/08/2021 20:13

Dallas Data Center schedule to maintenance scheduled Wednesday November 10th at 8PM PST.

Few servers will be effected with downtime in the process.

Dallas Datacenter maintenance (Resolved) Critical
  • 10/30/2021 10:00 - 10/31/2021 05:57
  • Last Updated 10/30/2021 15:19

Dallas Data Center schedule to maintenance tomorrow, Saturday morning - 10-11am CST.

All Dallas servers will be effected with downtime in the process.

c06 issue (Resolved) Critical
  • 09/02/2021 06:04 - 09/02/2021 08:13
  • Last Updated 09/02/2021 08:13

Hello,

 

We having issue with c06 that been reported. we are working to restored the services.

 

Update 1 : The issue has been fixed. Please if you still have issue submit ticket.

c15 LAX node issue (Resolved) Critical

Affecting Server - c15

  • 08/02/2021 16:16 - 08/02/2021 00:00
  • Last Updated 08/02/2021 17:05

We have issue with node c15 requiring quick rebooting.

 

Update : Server have connection crashing we are working into it.

 

Update 2 : Issue resolved.

New Jersey Maintenance (Resolved) Critical

Affecting System - NJ Servers

  • 05/22/2021 18:22 - 05/29/2021 21:50
  • Last Updated 05/27/2021 10:05

Greetings!

Update : All services online.

Update : Waiting fire up our network routed at new network.

Update : All servers has been moved We are waiting for network engineer to backup our prefixes.

On May 22, 2021 (9.00 AM EST to 9:00 PM EST)  we will be migrating all of our hardware from our provider in New Jersey to our new datacenter with Leaseweb. Our location still remains in New Jersey on the East Coast. we will continually update throughout the migration process. As we are physically moving hardware there will be some downtime associated with the migration. We expect downtime to last no more than 12 hours, however this is subject to change.

 

We appreciate your patience and understanding while we perform the migration to our new data center! Please do not hesitate to reach out to our support team if you have any questions.

 

Thank you!

New Jersey Network Maintenance (Resolved) Critical

Affecting System - Network Connection

  • 03/07/2021 19:54 - 05/19/2021 18:22
  • Last Updated 03/07/2021 19:59

We are reaching out to inform you about an upcoming scheduled network
maintenance to the core router which will cause service interruption . The major scheduled maintenance network is to increase a
big part of our network in New Jersey. 

Details -
Date: Monday, March 8th, 2021 / Tuesday, March 9th
Start Time: 6:00AM (GMT-6)
Work Window: Window : 6:00AM --11:00AM
Outage Estimated Duration: 30mins to 1 hour 
Effect: Major Scheduled Outage
Facilities: New Jersey, East Coast

If you have any questions about this maintenance or want more information on how
it will relate to your specific services with us, please feel free to reply to
this email.

Dallas, TX Network Maintenance (Resolved) Critical

Affecting System - Dallas, TX

  • 01/30/2021 06:52 - 02/26/2021 16:54
  • Last Updated 01/30/2021 06:56

We are reaching out to inform you about an upcoming scheduled network
maintenance to the core router which will cause service interruption . The major scheduled maintenance network is to increase a
big part of our network in Dallas. 

Details -
Date: Tuesday, February 4th, 2021
Start Time: 6:00AM (GMT-6)
Work Window: Window : 6:00AM -- 8:00AM
Outage Estimated Duration: 30mins to 1 hour 
Effect: Major Scheduled Outage
Facilities: Dallas, TX

If you have any questions about this maintenance or want more information on how
it will relate to your specific services with us, please feel free to reply to
this email.

C04 Cluster Maintenance LAX (Resolved) Critical
  • 01/29/2021 21:52 - 02/01/2021 19:59
  • Last Updated 01/29/2021 21:57


C04 Cluster Los Angeles scheduled Maintenance.


Details - 

Work Start Time: Saturday Jan 30, 2020  10:00 AM PDT / 8:00 PM GMT +2
Work End Time:  Saturday Jan 30, 2020  10:20 AM PDT / 8:20 PM GMT +2
Work Window: 1 to 2 Hours
Actual Outage Estimated Time: 20 minutes 
Effect: C04 Cluster Los Angeles Location


Thanks 

C14 Cluster LAX Location Maintenance (Resolved) Critical

Affecting Server - c14

  • 01/17/2021 10:00 - 01/18/2021 10:38
  • Last Updated 01/16/2021 21:43


C14 Cluster Los Angeles scheduled Maintenance.


Details - 

Work Start Time: Sunday  Jan 17, 2020  10:00 AM PDT / 8:00 PM GMT +2
Work End Time:  Sunday Jan 17, 2020  10:20 AM PDT / 8:20 PM GMT +2
Work Window: 1 to 2 Hours
Actual Outage Estimated Time: 20 minutes 
Effect: C14 Cluster Los Angeles Location


Thanks 

New Jersey MAINTENANCE NOTIFICATION (Resolved) Low

Affecting Other - Network Hardware upgrade

  • 01/21/2021 22:00 - 01/24/2021 18:35
  • Last Updated 01/16/2021 10:12

 

 

 

START TIME:

01/21/2021 10:00 PM EST

 

 

END TIME:

01/22/2021 02:00 AM EST 

 

 

WHAT:

Hardware Upgrade 

 

 

IMPACT:

None Expected

Subscribe our network status to received all (Resolved) Critical
  • 05/14/2017 00:47 - 01/15/2021 23:17
  • Last Updated 06/09/2020 20:40

network status to received all network updates and Maintenance

https://my.letbox.com/public-status.php?statid=1012cf5f68f40b43670a2694454f802a

Los Angeles Maintenance - September 10 (Resolved) Critical

Affecting System - Loa Angeles Network

  • 09/10/2016 02:00 - 05/14/2017 00:46
  • Last Updated 09/07/2016 23:00

Description:

 

The Why - 

We have seen an increase of high packet-per-second attacks compared to volumetric that cause degradation throughout the infrastructure. The upgrade will resolve specific areas that were exploited during these high packet-per-second application attacks. We are also taking the time to perform dark fiber maintenance by upgrading the capacity between all our locations as we have seen our traffic throughout triple within the year.

 

Though the work window may be long, the actual estimated outage time is only 10-15 minutes, but can happen at any time within the detailed window below: 

 

Details -

Work Start Time: September 10, 2016  2:00 PM PDT

Work End Time:  September 10, 2016 6:00 PM PDT.

Work Window: 4 Hours 

Actual Outage Estimated Time: 10-15 minutes or less 

Effect: Network Outage

Facilities: 624 Wilshire Blvd (1 Wilshire)

— Affecting Connectivity in 700 Wilshire Blvd (LAX2) & 609 W. 7th St. (LAX1), All Los Angeles, CA Clients

 

The long maintenance window will allow our engineers to roll back in the event of network degradation or outages. We’ve been preparing for this upgrade/maintenance for the last few weeks ensure things go as smoothly as possible and with as little downtime possible. 

 

If you have any questions on the process or any doubts please feel free to contact us anytime!

 

Thank you,

Los Angeles Network Maintenance (Resolved) Critical

Affecting System - Los Angeles

  • 06/19/2016 02:16 - 09/07/2016 22:53
  • Last Updated 06/15/2016 21:19

Description:

The focus of this maintenance will be to upgrade our Los Angeles Juniper core router’s operating systems which will require rebooting and cause a brief network outage.  We’ve run into several bugs in Junos OS (Juniper router operating system) which has hindered network performance recently, so for this reason we can no longer delay the upgrade to the latest stable version of the OS.

 

 

Details -

Date: Sunday, June 19, 2016

Time: 2PM PDT

Work Window: 3-5 Hours beginning at 2PM PDT

Actual Outage Estimated Time: 10 minute or less

Effect: Network Outage

Facilities: LAX1 & LAX2

 

If you have any questions on the process or any doubts please feel free to contact us anytime!

Los Angeles & Dallas Network Maintenance (Resolved) Critical

Affecting System - Cisco IOS

  • 10/09/2015 04:34 - 11/06/2015 11:45
  • Last Updated 10/10/2015 04:38

Los Angeles & Dallas. The focus of this maintenance will be to upgrade our network distribution routers to the latest versions of Cisco IOS. Though no outage exceeding 1 minute is expected, worst case scenario would be a 10 minute outage as the router reboots and loads the new version of the OS. 


Details -

Date: Saturday October 10, 2015

Time: 1AM CST through 5AM CST

Duration: No outage expected, but maintenance window for 4 hours while work is performed. 

Effect: Latest versions of IOS for distribution routers

Facilities: Dallas, Texas & Los Angeles, California

If you have any questions on the process or any doubts please feel free to contact us anytime!

LetBox servers scheduled maintenance (Resolved) Critical
  • 09/25/2014 11:00 - 10/10/2015 04:37
  • Last Updated 09/22/2014 01:51

we are scheduled maintenance LetBox Servers On September 25th 2014 at 11:00 PM GMT+2 We will performing scheduled maintenance on All LetBox Servers.  LetBox will experience an Internet outage. The duration of the outage will be approximately 15 minutes in aggregate.  By 11:40 PM GMT+2 all VPS Services reachability will be restored.

We apologize for any inconvenience. If you have any questions please do not hesitate to Contact us. Thank you for being a valued member of the LetBox family. Enjoy the rest of your day.

LetBox

Cisco Switch failure (Resolved) Critical
  • 10/29/2013 01:43 - 09/05/2014 20:12
  • Last Updated 09/05/2014 20:12

At 14:08 we receive failure with Cisco Switch on our Cabinet, We have tried to boot up our Switch but it ended badly but we did some action and use new switch.
However The new switch does not support IPv6 or 1 Gigabit port, We Plan to replace it to fit our needs and client needs next month.

Thank you for your patience.

Emergency kernel update (Resolved) Critical
  • 09/05/2014 19:57 - 09/22/2014 01:49
  • Last Updated 09/05/2014 19:59

Hello,


We will do emergency kernel update for node (D05 and D06). it will require one reboot the client will suffer between 10 to 20 minutes downtime, We are sorry.

Thanks.

Network scheduled maintenance (Resolved) Critical

Affecting System - Dallas Network

  • 09/07/2013 00:15 - 09/09/2013 23:19
  • Last Updated 09/05/2013 02:18

On September 7th 2013 at 12:01 AM CDT Dallas's dark-fiber provider will be performing scheduled maintenance on one half of the dark-fiber ring connecting the datacenter to the Internet.  LetBox will experience an Internet outage. The duration of the outage will be approximately 10 minutes in aggregate.  By 5:00 AM CDT all Internet reachability will be restored. 

Thanks You

LetBox Team.

RESCHEDULED Maintenance (Resolved) Critical
  • 05/04/2013 09:14 - 06/10/2013 04:26
  • Last Updated 05/04/2013 09:18

DATE: 05/06/2013 12:01 AM CDT

SPAN: 05/06/2013 12:01 AM CDT - 05/06/2013 1:00 AM CDT

DESCRIPTION:

On May 6th 2013 at 12:01 AM CDT CoreXchange engineers will be performing
scheduled maintenance on xxxxx's uplink.  Customers connected to
xxxx or customers connected to switches which uplink to
xxxxx will experience an Internet outage.  If you are receiving
this notification one or more of your uplinks involve cxswitch51.dllstx03.  We
expect the duration of the outage to be no more than five minutes, however
due to the nature of the work it is possible for the outage to last the entire
duration of the maintenance.  By 1:00 AM CDT all Internet reachability will be
restored.  If you have a redundant uplink there should be no interruption in
service.

Sincerely,

Dallas network maintenance (Resolved) Critical
  • 04/29/2013 09:16 - 06/10/2013 04:26
  • Last Updated 05/04/2013 09:17

On April 29th 2013 at 12:01 AM CDT CoreXchange engineers will be performing
scheduled maintenance on Dallas network customers will experience an Internet outage.  We
expect the duration of the outage to be no more than five minutes, however
due to the nature of the work it is possible for the outage to last the entire
duration of the maintenance.  By 1:00 AM CDT all Internet reachability will be
restored.  If you have a redundant uplink there should be no interruption in
service.

Sincerely,
LetBox

DDos attack (Resolved) Critical
  • 03/13/2013 21:14 - 03/13/2013 21:35
  • Last Updated 03/13/2013 22:17

We was suffering from a DDOS Attack with approximately 1.9gbit. The adress has been null routed and you should regain connectivity now.