CrowdStrike Falcon Sensor Glitch Causes Global Outage

In today’s ever-evolving society of constantly being connected, security is of the utmost importance. Software and security patches are a key part of keeping technology up-to-date and protected. Recently, many in the world, especially those in the banking and travel industry, faced a massive tech outage caused by a software issue with a security update by cybersecurity company, CrowdStrike. 

Who is CrowdStrike?

CrowdStrike, the developers of Falcon (a software designed to protect networks), is currently the number one company for endpoint security. This protects items connected to the end of the network, i.e. computers or workstations etc., that are exposed to the internet directly and could be compromised to allow unauthorized access to a network. This unauthorized access can expose an entire business if the network is not secured, and bad actors gain access.

What Happened During the CrowdStrike Falcon Software Glitch

On July 19, 2024, at 4:09 UTC, CrowdStrike released a sensor configuration update to Microsoft Windows systems for the Falcon software. This update triggered a display error of 0x50 or 0x7E Blue Screen of Death (BSOD) and/or a constant system reboot, known as a “boot loop,” both of which caused Microsoft Windows devices not to boot to the Operating System (OS). The affected computers were not able to operate as designed and were unusable. 

The sensor configuration update was remediated by CrowdStrike the same day at 5:27 UTC, meaning the faulty update was available for just over one hour and fifteen minutes. It is estimated that 8.5 million were impacted globally by this issue.

View the full details and timeline.

Mitigating the CrowdStrike Falcon Software Glitch

Many businesses in the Information Technology (IT) industry were quick to identify the cause of the problem, identified as a Channel File 291 issue. The fix was to remove a file (ending in 00000291.sys) in the C:\Windows\System32\zdrivers\Crowdstrike directory specifically. Once this file is deleted, and the computer is rebooted, the issue should be resolved. 

Access to this file requires a boot to Safe Mode or the Windows Recovery Environment (Windows RE). Safe mode and Windows RE are maintenance modes for diagnostics and repairs. Safe Mode loads a small subset of software and drivers to enable users to perform tasks where the full software and driver load may have contained the issue, such as in this case. Windows RE is built into the Microsoft OS and is a tool to allow recovery of a system that is not able to boot to the OS. It offers a set of tools to perform similar tasks but with more features. 

How to Remediate the CrowdStrike Falcon Software Glitch

If you are affected by a Windows BSOD error that may be associated with CrowdStrike, use the following temporary actions: 

1. Boot Windows into Safe Mode or the Windows Recovery Environment
2. Navigate to C:\Windows\System32\zdrivers\Crowdstrike directory
3. Locate the file ending in 00000291.sys and delete it
4. Boot the PC again normally

Additional Resources:

• Falcon Content Update Remediation and Guidance Hub
• Microsoft New Recovery Tool to Help With CrowdStrike Issue Impacting
• Using the Microsoft Recovery Tool for Automated Host Remediation

CrowdStrike Falcon Faulty Software Update

The Falcon configuration update was faulty and did not cause any physical damage. Rather it triggered an unforeseen interaction with the Microsoft OS platform. Additionally, only users who downloaded the update manually or automatically between the time it was available would be susceptible to the crash. Unfortunately, the automated updates that many utilize would have downloaded the Falcon update if the equipment were online and connected to the internet, triggering the issue. 

Automatic updates are an idle way to stay compliant with software and security updates but come with risks such as this.  Many large businesses know software updates from Microsoft and other publishers can wreak havoc and will, in turn, perform software updates in a test environment before a production deployment. However, some businesses have automatic updates turned on, hence this issue. 

While the temporary fix is fairly simple overall, downtime was and will be incurred while staff address each impacted item. Microsoft’s recovery tool will allow for a more automated recovery, but business interruption will be based on response and the number of impacted equipment. Typically, a standard response would have all equipment returned to normal use within 48 hours.