I have to get out NOW from my autonomous vehicle: urgent egress and passenger overrides

What if you need to get out of the vehicle RIGHT NOW in a robotaxi — is that allowed? What are the implications?

In any automated system there will be times when an
occupant wants to override the automation, and especially when they want to
exit a moving automated vehicle. Reasons might include: wanting to re-open
transit vehicle doors if a passenger was unable to exit in time at their stop; an
attack of claustrophobia; wanting to get away from another passenger due to
personal safety concerns; or even needing to escape a cabin fire. Some egress
requests might constitute misuse or abuse, such as stopping a vehicle to
intentionally block traffic, or intentionally accessing an off-limits area such
as a bridge with no pedestrian infrastructure.

Creating a complete list of all possible
motivations is difficult, and weighing the merits of all such egress attempts
in advance seems intractable. Nonetheless, there are times when a passenger
desire to exit a moving vehicle should be honored, although the vehicle should
likely at least stop before permitting an exit.

In still other situations passengers might want to
force an otherwise stopped vehicle to move. One reason might be fear for
personal safety if threatened by malicious actors while stopped at a traffic
light. Another reason might be overriding a police stop if the vehicle occupant
suspects a stopping officer is instead a criminal imposter, at least until
legitimacy of the police stop can be confirmed via contact with an emergency
dispatcher.

Another special situation is one in which a
passenger has a compelling reason to order an AV to operate outside its ODD or
with degraded equipment in an emergency, even if doing so will result in a
reduced safety margin. For example, an AV might be programmed not to drive
through heavy smoke, but doing so might be required to escape a burning town in
a wildfire situation. The AV occupant might want to take the chance of driving rather
than remaining in the burning town.

Human drivers have the authority to deal with these
situations so long as they are willing to accept the responsibility. Do you
start driving when someone is trying to forcibly break into your car at a
traffic light even if you might injure that malicious actor by doing so? The
choice – and responsibility for consequences – falls upon the person driving a
manually driven car.

The question is: to what degree should an AV
support operator overrides of safety-relevant behaviors? A complication is that
there might not be a responsible individual in a vehicle to exercise control.
What if a passenger is allowed to override some behaviors of the vehicle, but
that passenger is impaired, or not capable of exercising mature judgment?
Should an 8 year old riding solo be able to command vehicle safety overrides?

Answers as to how much control a passenger should
have over AV operation will depend on how stringent qualifications are for a
passenger to be capable of mature decision making. It is easy to say there must
be one qualified driver if there are any passengers in an AV and that manual
controls must be available if needed. However, requiring a qualified driver
undermines the potential benefits that AVs might provide for those who are not
capable of driving or should not be driving at a particular time.

If other than unimpaired licensed drivers are
permitted to override AV behaviors, there will be difficult tradeoffs as to
what overrides might be permitted. Likely an 8 year old child should not be
permitted to exit a school vehicle in the middle of a highway to avoid going to
school. On the other hand, a 14 year old might
be considered mature enough to demand an emergency stop if the vehicle tries to
drive into flood waters, or initiate an emergency exit with their younger
sibling if the cabin fills with smoke from a vehicle battery fire.

Even if a passenger is an adult licensed to drive,
should that adult be permitted to override vehicle behavior if drunk or
otherwise impaired? If not, should the vehicle disable override capability if
the passenger is drunk? Or should it be illegal to enter an AV with override
capability when drunk?

While it can be an interesting exercise to conjure
extreme situations, the issue of passenger overrides and egress can also be as
simple as a passenger saying “I want to get out now” when the vehicle is
stopped at a red traffic light but not at the end of the scheduled trip. Should
the passenger be able to unlock doors and exit? Or should the passenger be kept
locked inside the vehicle until the end of the trip? Should there be a
workaround available such as changing the destination? If so, should the
passenger have permission to do this if some authority figure such as a parent
input the original destination? Where should the threshold be drawn at which
such a passenger request is denied both in context (speeding down a highway vs.
stopped) or passenger maturity (a passenger one day before turning 18 years old
but with no driver’s license vs. grade school age child)?

There is the possibility that remote operators will
need to mediate requests for overriding AV behavior either routinely or if
there is doubt as to the competence of passengers to make reasonable decisions.
However, any such remote operators can be expensive, will have problems
scaling, and might result in wait times long enough to impair safety by
delaying decisions in urgent situations.

For AVs to be deployed at scale, designers will
need to decide how much authority passengers have to override vehicle behavior,
and whether emergency manual vehicle controls will be required even in vehicles
that are intended to be completely automated. There will be no perfect policy choice,
but not setting a consistent policy is also a policy choice.

Thanks to Fred Perkins from the Center for Auto Safety for bringing this topic up in discussions on UL 4600.

This is an adapted excerpt (Section 10.4.4) from my book: How Safe is Safe Enough? Measuring and Predicting Autonomous Vehicle Safety