IT Equipment Common Perils: Abrupt Loss of Power

It is common for utility power to be interrupted several times a year for most consumers in the US. An interruption of utility power is one of the most common causes of problems with IT equipment, which can last for a few seconds, minutes, or an extended period. The cause of a power interruption can vary, but common culprits include equipment failure, scheduled maintenance work by the service provider, and damage to the power grid during a storm or hurricane. Often, this results in not just one, but multiple power loss events before power becomes stable.

Any loss of power can cause IT equipment to become inoperable. For computers, servers, storage devices, and other network devices, a power loss results in an abrupt loss of power to this equipment and leads to what is known as a hard or non-graceful shutdown. An abrupt power loss to IT equipment can cause software, data, and configuration corruption. 

What is Software, Data and Configuration Corruption?

Corruption in IT equipment does not constitute physical damage but rather a logical issue. The corruption may be presented to the end user as an error during the equipment starting, a blue screen message, files not opening, continuous rebooting or other abnormal behavior. Corruption can often be overcome by reinstalling software, backups of data, and/or reconfiguring the equipment. Unfortunately, IT equipment has software that is stored within onboard internal semiconductor components that may not be readily restorable, meaning replacement of the affected component may be necessary. The most commonly seen component affected by corruption is the motherboard for the computer or server. 

A power loss can also reveal worn components as power cycling or rebooting equipment causes the IT equipment to perform a verification upon startup that detects anomalies, such as components outside of manufacturer specification. As a result, users are not aware of the component defect until the system is rebooted or power is cycled.

Often the most disastrous issue is not the fact that the equipment is not working properly, but that the data hosted on the IT equipment or the function is not available, resulting in lost data and downtime. Ensuring data and configuration backups are working as designed or performing preventative maintenance regularly are wise steps to prevent failures and loss of important data. Not only ensuring that the backups are occurring but also confirming the backups have the required data is often overlooked and can result in disaster. 

Imagine discovering after a loss of data on a server that the information you thought was being backed up was not complete or worse yet, not able to be restored.

Data Backups to the Rescue

Regardless of the cause of a power loss, if it is not preventable, vital company data must be verified to be on a known reliable backup system. There should be multiple backups where at least one set is stored offsite in a secure location; backups should not be stored on the same device that the original data is stored on. Backups can span over the last few hours or even the past week, month, or more.

As a rule of thumb, backups should be tested and the data should be verified that it is available and can be accessed after a restore process completes. Verifying that data is available can provide peace of mind. A safety deposit box, cloud storage, or another office location can ensure that this vital asset is protected and available should disaster strike. 

Types of Data Backups 

• Full Backup: As the name implies, everything is backed up. This requires the most storage capacity and time to complete. 
• Incremental backup: Only backs up data that has changed or is new since the last backup. Following an initial backup, the new or changed data is backed up as a different set of data. If the initial backup does not restore, the other incremental backups are not usable.  
• Differential Backup: Only backs up data that has changed since the last backup. This is similar to the Incremental backup, but only data that changed is backed up. Again, the original backup must be operational for data restoration to be successful. 

What Can Be Done to Prepare for a Power Loss? 

With sufficient design and preparation, damage or corruption from a power loss can be prevented or mitigated entirely.  If there is not sufficient infrastructure to protect IT equipment, backing up the data and powering off equipment properly before a known power loss or a major storm/hurricane is suggested.

If there is sufficient infrastructure, such as battery backups, also known as Uninterruptible Power Supplies (UPS) and a backup generator, vital equipment can be supplied power during extended interruptions of utility power by these devices. In this scenario, the UPS prevents a loss of power to the equipment in the short term until the generator can start, stabilize, and switch over to provide long-term, stable power to the equipment. 

It is important to complete proper maintenance of the equipment, which should be replaced every three to five years. All batteries should be replaced at the same time, not just a few at a time. Verifying a generator is operational and has sufficient fuel along with performing maintenance to the generator is important as well. A test of the operation of this equipment periodically during a maintenance window is recommended, as this verifies the operation of this vital infrastructure. These routine maintenance items and steps can often prevent an abrupt loss of power, failure of equipment, and crucial loss of business data. 

If a UPS is being utilized with no power generator, a very common setup, the UPS often has specialized software that can be installed and configured to gracefully power down equipment once the battery charge drops below a certain percentage or number of minutes remaining. Some UPS models can also issue an alert to IT staff to notify them that the systems are operating on backup power.

Critical IT equipment should not be provided with power only by a surge protector or power strip. These devices do not provide any protection during a power loss. In addition, some UPS models have both surge-only and battery-plus-surge receptacles. The surge-only receptacle supplies no battery during an interruption in power but does offer surge protection, while the battery-plus-surge has both battery and power surge protection. Critical equipment should be plugged into the battery-plus-surge receptacles to prevent this common peril. 

Backup strategies, infrastructure resilience, and regular maintenance are essential to prevent equipment failure and safeguard against the potential loss of valuable business data. Proactive measures taken today can significantly mitigate the impact of power interruptions, ensuring the continuity of operations and data integrity in the face of unforeseen challenges.