Search
Close this search box.

Fast CrowdStrike Recovery should be a reality for Cristie Data customers

Share This Post

Fast CrowdStrike Recovery should be a reality for Cristie Data customers.

Following the CrowdStrike bug that triggered a global IT outage on July 18th and 19th, 2024, CrowdStrike recovery has become a critical activity and focal point for many organizations. While the exact number of impacted users remains unknown, news reports offer some valuable insights:

 

  • Widespread Disruption: News articles describe the outage as widespread, disrupting airlines, financial institutions, hospitals, and businesses.
  • Large Organizations: The affected entities suggest a significant number of users, potentially impacting thousands or even millions of devices.

 

The CrowdStrike update bug sent shockwaves through the IT world, causing major disruptions for countless organizations.

What caused the CrowdStrike Falcon update outage?

The outage was caused by a faulty update for the Windows version of their Falcon sensor.
Here’s a breakdown of the issue:

 

  • Falcon Sensor: This is a core component of the CrowdStrike Falcon platform that runs locally on user devices and scans them for potential malware threats.
  • Faulty Update: A specific update for the Windows version of the Falcon sensor contained a defect.
  • System Crash: This defect triggered a “logic error” that caused the affected systems to crash and enter a continuous restart loop, often referred to as the “Blue Screen of Death” (BSOD) on Windows machines.

 

Here’s some additional information:

 

  • Limited Scope: The issue only affected the Windows version of the Falcon sensor, not Mac or Linux systems.
  • Recovery: CrowdStrike identified the issue and deployed a fix. However, restarting the affected systems multiple times might have been necessary to complete the recovery process.

 

While CrowdStrike hasn’t officially pinpointed the exact cause within the update, available information suggests a software error triggered the system crashes.

How can Cristie customers recover faster from the CrowdStrike update crash?

The CrowdStrike update recovery process presents a perfect use case for automated system recovery. In this explainer video, Sky News business correspondent Paul Kelso outlines the time-consuming manual process required to recover systems to allow for deletion of the disruptive CrowdStrike driver file. Users with large server estates that do not utilize automated system recovery or boot management tools would face a significant amount of manual intervention and downtime in order to facilitate driver removal from all effected machines. Cristie Software bare machine recovery (BMR) provides system recovery that works directly with leading backup solutions such as Rubrik Security Cloud, Cohesity DataProtect, IBM Storage Protect and Dell Technologies backup solutions Avamar and Networker. Using Cristie recovery software automation, the following steps would be required to recover effected machines to a point before the disruptive CrowdStrike driver was applied:

 

 

• Reboot in DR environment: Reboot systems into DR environment (this can be automated using boot management tools with our web-boot ISOs).
• Recover systems to last know good point in time: Trigger recovery from backup server (Rubrik, Cohesity, IBM or Dell).
• Reboot system: Machines would reboot to the last known good state prior to the application of the disruption driver.

In addition, data protection solutions offered within the Cristie READY program offer automated system recovery to a specific point in time which could also facilitate automated recovery from operating system level failures such as the CrowdStrike update bug.

Fast recovery for multiple VMs with Cristie Cyber Recovery powered by Rubrik

If you are a Cristie Data customer affected by the CrowdStrike update bug on July 18th, 2024, and you use our Cyber Recovery powered by Rubrik with immutable backups, here’s how to recover your virtual machines (VMs):

Benefits of Using Rubrik:


• Faster Recovery: Recover Windows VMs (VMware) significantly faster by only restoring changed blocks, reverting them to a snapshot before the problematic update (04:09 UTC).
• Large-Scale Recovery: Quickly bring your environment back online by reverting large groups of VMs to a working snapshot in minutes.


Important Considerations:


• This recovery process will restore your VM to a previous point in time (before the update). Any changes made to the system since then will be lost.
• If you’re unsure about the recovery process, Cristie Data support is available to guide you through VM recovery.

 

For a faster recovery experience, consider using Rubrik’s in-place recovery or live mount options, depending on your VM type.


Remember: Cristie Data support is always available to assist you with any questions you may have regarding VM recovery.

What is the manual CrowdStrike update recovery process?

Following the CrowdStrike Falcon update bug, recovery methods varied depending on the severity of the issue and system access. Online resources outline two main approaches: Booting into Safe Mode or the Windows Recovery Environment (WinRE) for continuously restarting systems, and detaching the disk from a virtual server (for advanced users).  However, due diligence is crucial.  Always consult CrowdStrike support to verify the appropriate recovery procedure for your specific environment to ensure a successful and secure restoration:

 

  1. Booting into Safe Mode or Windows Recovery Environment (WinRE):

This method was recommended by CrowdStrike for situations where the system continuously rebooted into a loop (BSOD).

 

Here’s how it worked:

  • Boot into Safe Mode: This can be achieved through various methods depending on your system configuration. One common approach is to repeatedly press the F8 key during system startup.
  • OR Boot into WinRE: If Safe Mode is inaccessible, you can try booting into the Windows Recovery Environment (WinRE). This may involve using a bootable USB drive or recovery media provided by your system manufacturer.
  • Navigate to the Target Directory: Once in Safe Mode or WinRE, locate the folder containing the problematic CrowdStrike files. The specific path might vary, but it’s typically something like C:\Windows\System32\drivers\CrowdStrike.
  • Delete the Faulty File: Look for a file named “C-00000291*.sys” (the asterisk represents any wildcard characters) and delete it.
  • Reboot Normally: After deleting the file, attempt to reboot your system normally. If the issue was resolved, the system should boot up successfully.
  •  
  1. Detaching the Disk from a Virtual Server (Advanced Users):

This option was suitable for virtualized environments where the affected system was running on a virtual machine (VM).

 

It’s important to note that this method requires technical expertise and should only be attempted by experienced users.

 

Here’s a simplified overview:

  1. Detach Disk: Detach the virtual disk volume from the impacted virtual server. Create a backup or snapshot of the disk volume as a precaution.
  2. Mount Disk on Another Server: Attach or mount the detached disk volume to a separate virtual server with a working CrowdStrike installation.
  3. Access and Delete File: Follow steps similar to the Safe Mode method to access the C:\Windows\System32\drivers\CrowdStrike directory and delete the “C-00000291*.sys” file.
  4. Reattach Disk and Reboot: Detach the disk from the temporary server, reattach it to the original impacted virtual server, and attempt a normal reboot.

Additional Tips:

  • Consult CrowdStrike Support: If you’re unsure about the recovery process or encounter difficulties, it’s advisable to reach out to CrowdStrike support for assistance.
  • Test Functionality: Once your system boots up successfully, verify that your CrowdStrike Falcon sensor is functioning correctly.
  •  

Remember: These guidelines are general and gleaned from online resources.  The specific steps may vary based on your system configuration and the severity of the issue. If unsure, always consult a qualified IT professional for assistance.

 

The CrowdStrike update fiasco highlights a critical gap in many companies' disaster recovery plans: system recovery.

While data backups are essential, they often fall short when a system-level issue, like a faulty driver update, prevents the system from booting altogether. This leaves companies scrambling to rebuild systems from scratch, a time-consuming and error-prone process.

 

 

Here’s how the Cristie BMR Suite steps in to bridge this gap:

 

 

  • Complete System Recovery: It goes beyond data backup, capturing operating system configurations and enabling restoration of entire systems to any point in time.
  • Automation for Efficiency: Cristie BMR Suite automates the physical machine recovery process, eliminating the need for manual intervention. This translates to significant savings in administrative overhead, especially during large-scale server recoveries.
  •  

The CrowdStrike incident serves as a wake-up call. By implementing a comprehensive system recovery solution such as the Cristie BMR Suite or a Cyber Recovery solution from the Cristie READY program, businesses can ensure they’re prepared for any eventuality, minimizing downtime and maximizing operational resilience.

 

 

Contact the Cristie Data-Team if you have been affected by the CrowdStrike update failure and would like to learn more about system recovery and recovery automation.

Protect Your Organization with Arctic Wolf

Protect Your Organization with Arctic Wolf’s IR JumpStart Retainer Program: A Proactive Approach to Cybersecurity In today’s evolving threat landscape, organizations are more vulnerable than ever to cyberattacks. The need for a swift, expert-led response to cyber incidents is crucial to limiting damage and ensuring recovery. Arctic Wolf’s Incident Response

Recent Cyber Attacks on Healthcare Organizations in Germany: A Growing Threat

Recent Cyber Attacks on Healthcare Organizations in Germany: A Growing Threat In our recent article we highlighted the top 5 industries targeted by cybercrime with the healthcare sector being number one. Germany’s healthcare sector has become a prime target for cybercriminals, with several significant cyberattacks in recent years causing widespread

The 5 Industries Most Targeted by Cybercrime

The 5 Industries Most Targeted by Cybercrime Cybercrime has become one of the most significant threats to global businesses, with hackers and cybercriminals increasingly targeting industries where sensitive data, critical infrastructure, and valuable financial information are stored. As the world becomes more digitally interconnected, cybercriminals are becoming more sophisticated, exploiting

NIS 2 and KRITIS: What Companies Need to Know

NIS 2 and KRITIS: What companies need to know NIS 2 (Network and Information Security Directive 2) and KRITIS (Critical Infrastructure Protection in Germany) are both regulatory frameworks designed to enhance the security and resilience of critical infrastructures, but they differ in scope, geographic focus, and specific requirements. Here’s a

How Does Cohesity Protect Organizations from Ransomware Attacks?

How Does Cohesity Protect Organizations from Ransomware Attacks? In an era where ransomware attacks have become increasingly sophisticated and frequent, organizations need robust solutions to safeguard their data. Cohesity which Cristie Data customers can benefit from within the Cristie READY program offers a comprehensive approach to protecting against ransomware by

How Does Cohesity Simplify Backup and Restore Processes?

How Does Cohesity Simplify Backup and Restore Processes? Cohesity Backup Solutions: Streamlining Your Backup and Restore Nightmares into a Manageable Dream In today’s data-driven world, safeguarding your information is paramount. Traditional backup solutions can be complex, time-consuming, and struggle to keep pace with ever-growing data volumes and cybersecurity demands. Here’s

it-sa 2024
Request your FREE admission voucher code.

Thank you for your registration!






Teilnahme nach Verfügbarkeit.

CMT24 - Registrierung
Cristie Mopped Tour 2024

Thank you for your registration!






Teilnahme nach Verfügbarkeit.

Schauen Sie sich das DORA Video an

Thank you for submiting your email address. Press the button below to download the pdf.

Watch the Spectra Tape Video

Thank you for submiting your email address. Press the button below to download the pdf.

Schauen Sie sich das NIS2 Directive Video an

Thank you for submiting your email address. Press the button below to download the pdf.

Save the Data - Event Registrierung

Thank you for your registration!






Teilnahme nach Verfügbarkeit.

Arctic Wolf - Security Breakfast

Thank you for your registration!





Teilnahme nach Verfügbarkeit.

Arctic Wolf - Security Breakfast Event

Thank you for your registration!





Participation subject to availability.

eBook: Transform Your Business with Mature Data Management

Thank you for submiting your email address. Press the button below to download the pdf.

Understanding LTO-9 Tape Technology – Whitepaper

Thank you for submiting your email address. Press the button below to download the pdf.

Understanding LTO-9 Tape Technology – Whitepaper

Thank you for submiting your email address. Press the button below to download the pdf.

Contact Info

Nordring 53-55, 63843 Niedernberg,
An der Burg 6, 33154 Salzkotten,
Germany

Monatliches Angebot für Cloud-Schutz anfordern

Thank you for your registration!

Wählen Sie mehrere aus, indem Sie beim Auswählen die Taste strg oder cmd drücken.

*Sie können die Anzahl der zugewiesenen Lizenzen in Microsoft 365 ermitteln, indem Sie zur Seite Microsoft 365 Admin center > Billing > Licenses navigieren.

** Die folgenden Abonnements werden von Cristie Cloud Backup für Google Workspace nicht berechnet:
Google Voice Starter (SKU ID: 1010330003)
Google Voice Standard (SKU ID: 1010330004)
Google Voice Premier (SKU ID: 1010330002)

Auf dem Weg zur intelligenten Welt – Whitepaper

Da neue Technologien wie 5G, IoT, Cloud Computing und Big Data in der digitalen Transformation eingesetzt werden, bewegt sich die IT-Architektur von Unternehmen in Richtung eines hybriden Frameworks aus „traditioneller IT + privater Cloud + öffentlicher Cloud + Edge“.

Thank you for submiting your email address. Press the button below to download the pdf.

Striding Towards the Intelligent World – White Paper

As new technologies, such as 5G, IoT, cloud computing, and big data, are being applied in digital transformation, enterprise IT architecture is moving towards a hybrid framework of “traditional IT + private cloud + public cloud + edge”. This report provides an in-depth outlook on the development of the data storage industry.

Thank you for submiting your email address. Press the button below to download the pdf.

Zero Trust Data Security for Dummies

Thank you for submiting your email address. Press the button below to download the pdf.