Following acceptance of blame, lawmakers impressed by CrowdStrike's 'humility'

Two months after a buggy update caused widespread IT outages across the globe, the cybersecurity firm that caused it faced both praise and scrutiny from U.S. lawmakers looking for details on how exactly the outage occurred.

CrowdStrike sent Adam Meyers, senior vice president of counter adversary operations, to testify before the House Cybersecurity and Infrastructure Protection Subcommittee on Wednesday in a 90-minute hearing that yielded the company more praise than condemnation from lawmakers.

The hearing came after a July 19 incident that prevented numerous Windows users from logging into their computers, including employees at Fifth Third Bank. At TD Bank, online and mobile banking were disrupted. Synovus Financial had to implement “contingency plans” to minimize disruptions to clients. All branches and bank offices of Canandaigua National Bank, a $5 billion institution in Canandaigua, New York, were affected.

Other sectors were hit even harder. Airlines experienced a spike in delays and flight cancellations at the onset of the errors just after midnight East Coast time. Delta ended up canceling 7,000 flights and suffering losses of $550 million. NBC News, Sky News and several Australian broadcasters temporarily stopped broadcasting live content.

“The sheer scale of this error was alarming,” said Andrew Garbarino, R-N.Y., chair of the subcommittee. “If a routine update could cause this level of disruption, just imagine what a skilled, determined nation state actor could do. We cannot lose sight of how this incident factors into the broader threat environment.”

Many of the most dramatic interruptions were resolved within a day; it took 10 days for error rates to return to the pre-incident normal, according to CrowdStrike.

During the hearing, multiple lawmakers including Laurel M. Lee, R-Fla., focused on a change CrowdStrike made after the debacle: enabling phased rollouts for security updates rather than pushing updates to all customers at the same time. Combined with improved testing, the move is meant to reduce the risk of widespread outages in the future.

During the hearing, Lee asked Meyers whether he agreed that failing to implement phased rollouts of rapid response content “ended up being catastrophic.” He said the company was putting “a lot of time and effort” into ensuring that customers will have the ability to choose when and how they receive such updates.

Lee and others also questioned Meyers about CrowdStrike software’s kernel-level system access, echoing concerns from some observers that CrowdStrike’s use of a kernel driver must be weighed against the risks of totally crashing the computer, rather than crashing just the CrowdStrike application, in the case of an error.

Meyers said he could not think of a security product that doesn’t have a kernel driver. One reason for this is that endpoint detection and response, or EDR, software, which monitors computers for fishy behavior and stops it once detected, must have access to the whole system to detect threats, otherwise threat actors would just target the software’s blind spots.

Lawmakers were not strictly critical of CrowdStrike’s response to the incident. At points, they explicitly praised the company for its response.

One element that received praise was CrowdStrike’s apology. CEO George Kurtz initially made a statement about the incident that did not express contrition, which earned him some flak in the media. But, by the end of the day, he said sorry.

“I want to sincerely apologize directly to all of you for the outage,” Kurtz said in a statement posted on CrowdStrike’s website on July 19. “All of CrowdStrike understands the gravity and impact of the situation. We quickly identified the issue and deployed a fix, allowing us to focus diligently on restoring customer systems as our highest priority.”

This week, Meyers made his own apologies in written testimony before the subcommittee, adding that the company appreciates the “round-the-clock efforts” of customers and partners who “mobilized immediately to restore systems and bring many back online within hours.”

The sentiment impressed Mark Green, R-Tenn., chairman of the House Committee on Homeland Security.

“There was a degree of humility there that is impressive, and I appreciate the transparency that we have seen,” Green said. “I think some of the biggest lessons we learn are in times of adversity, and you guys have shown the right attitude. So, thank you.”

Green’s colleague Tony Gonzales, R-Texas, a member of the subcommittee, echoed his sentiments, saying he was “grateful” for CrowdStrike’s rapid response and the documentation they released publicly to explain the error.

On the other side of the aisle, Democratic lawmakers were not as overtly warm toward CrowdStrike. Eric Swalwell, D-Calif., the top-ranking Democrat on the subcommittee, said in his opening remarks that the subcommittee was “not here today to malign CrowdStrike,” but rather to get to the bottom of the circumstances and failures that led to the outage.

Beyond Swalwell’s comments, Democratic members thanked Meyers only for appearing before the committee.