Waymo’s Misleading Claim of Saving Lives

Waymo claims they are “already saving lives” so often, and people are so taken in by that misleading claim, that I’m going to take a moment to explain why it is misleading. And especially harmful when used as justification for loose regulatory policies as it so often is.The claim:  “The data to date indicates the Waymo Driver is already reducing traffic injuries and fatalities.”   Here is the claim, which has been at the top of the Waymo Safety landing page for quite a while now (https://waymo.com/safety/  as of June 4, 2024;  highlighted of those words added):

Having had high school English, I would interpret that sentence as also including an unproven claim of “already reducing fatalities”.   And I would expect that anyone authoring this sentence would reasonable expect a reader or listener to conclude “already reducing fatalities.”  Those listeners include federal and state regulators and legislators.

This claim is absurd for a simple reason.  US driving data shows human-driven vehicles have ballpark 1 fatal crash per 100M miles (varies by year, zip code, etc. — for more nuance see this narrated video slide which is in terms of fatal crashes, noting that some such crashes have multiple fatalities).  But their latest study is for only 7.1 million miles. They need something like 40 times more data prove they are actually saving lives with statistical confidence (likely even more).

What is really going on here is some sort of word game that is essentially guaranteed to mislead readers. Their 7.1 million mile study talks about a bin called “any-injury-reported” crashes that were lower than human-driven vehicles, and fatalities are a subset of that bin.  So the claim being made is (apparently) the bin containing fatalities is better than human drivers. Without mention that the sample size is too small for valid conclusions on fatalities.  So maybe they have saved about 0.07 or perhaps even 0.10 lives depending on the baseline — and maybe not.

But don’t just take my word for it, see for yourself this excerpt from Waymo’s own paper saying “Serious injury and fatalities are a subset of this any-injury-reported benchmakr, but no statement on these outcome levels can be made at this time based on this retrospective data.”  In other words, Waymo does not have enough data to know how fatalities will turn out.  That’s the truth. Waymo’s safety landing page claim is something other than the full truth.

Waymo paper:  “Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles”  https://arxiv.org/pdf/2312.12675  (top of page 15; highlight added)