Waymo AMA concluded. Rather that copy and paste the entire thing, here are a few questions that I thought had interesting answers:
I also wonder how Waymo deals with debris and potholes. Does it know how to swerve or will it just drive over them?
Yes! Debris and potholes are detectable. The Waymo Driver considers the nature of the object / pothole, and makes plans around it. Although the Waymo Driver has swerving capabilities, its actions would be very scenario-specific. - John
What do you see as the path forwards for validating these large foundational models? Will we see them adopted for AVs in the near future or will the safety considerations prevent their adoption?
Hey, it's Kris. The general public is becoming more aware of the advances in generative machine learning models, but Waymo has been carefully and strategically using this type of approach for additional robustness in our perception, behavior prediction, and planning for quite some time (check our AMA with Drago from Waymo’s ML Research team from 2020). As a safety researcher, I’m not an expert in developing these types of models, but our safety readiness determination process has been developed to assess a system that contains learned models from the beginning. So, I’m not going to make any predictions about what specific approaches Waymo may take in the future, but I’m confident our safety framework can evaluate the overall safety of these components as we evolve our driver.
How often do rides in San Francisco require intervention by a remote operator (e.g. per X passenger miles)? Has this metric improved over the past two years?
Remote assistance is an industry standard part of full-stack AV operations, but the definition of ‘remote assistance’ varies from company to company. This is why I won’t be sharing numbers. But I want to provide more context on how it works at Waymo.
We have a fleet response team that works remotely that we like to think of as air traffic control for our autonomously driven cars. If the Waymo Driver encounters a novel or ambiguous situation, it can request additional info. But it generally does not rely on this input from the fleet response, nor does it wait for that input. That is because most of these instances are resolved by the Waymo Driver without assistance, even before the fleet response team responds.
Can you give a hint to international plans?
Sure, I can hint: We have plans but they're further out. We're focused on the US for now, though I can't wait for the day I can get around Edinburgh (where I am now) in a Waymo!
#mycitypleasewaymo - Trent
1. I have taken waymo a few times in SF and I found the waymo driver to be quite impressive. But the pick ups and drop offs were not great on some trips, the car would take longer to leave or would park itself in an awkward spot. Does waymo look into this as well from a safety point of view?
I have seen cars and bicycles going around waymo when it is parked putting them maybe at a slightly higher risk.
2. When waymo decides to onboard a new city, Is there a localization process for safety protocols too or does it universally apply across cities without major changes for each new city?
For (1) We absolutely aim to be a considerate driver – what we call “drivership.” In this sort of scenario, you need to consider (a) severity potential, (b) how your actions influence the safety of other road users, and (c) how your actions affect your occupants. On a high-level, Waymo considers frequency, severity, and avoidability in behavioral evaluations. The scenario you highlighted is one in which safety evaluation is relevant and would be applied, but I am not sure about the specific circumstances you have personally observed.
For (2), there are certainly novel challenges in each new geographic area / ODD update. We identify novel challenges through our on-road testing, simulation, and various other crash / naturalistic driving data sources. Requirements are then set in accordance with any novel challenges we might expect / observe. - John
What do you see as some of the safety challenges to deploying rider-only on interstate highways versus deploying rider-only on city streets?
As we pointed out in our
recent human benchmark paper, not all roads are created equal when it comes to safety. Freeways tend to have fewer overall crashes per mile traveled compared to surface streets, but if a collision does happen there, it’s more likely to involve higher speeds and greater forces. We’re in our
early stages of deploying rider-only on freeways. To get to this point, we’ve had to adapt our safety readiness determination for freeways. The good news is a lot of what we’ve learned from driving on surface streets translates over to freeways. But just like with our initial deployments on surface streets, we are going to take a measured approach to build up confidence in our safety assessments before further expanding our operations. - Kris
Can you say if you anticipate validation of safety will be quicker for future cities? Once you saturate existing markets, I’m wondering if it’ll be a multi year wait for new markets. Any update on Austin in terms of progress validating safety?
Hey, it's Kris. As we pointed out in our recent human benchmark paper, not all roads are created equal when it comes to safety. So there are some things (like the benchmark crash rate) that should be considered for each new city. That said, the Waymo Driver is designed to generalize across different driving environments. As we’re entering new cities, we are seeing that the Waymo Driver does generalize well, so we are expecting that the effort to expand to city 3 and 4 (and beyond) will not be the same as the effort to go from city 1 to 2. Freeways are similar (we’re in our [early stages of deploying rider-only on freeways] in that a lot of what we learned driving on surface streets carries over, but, of course, there are new challenges. As we’ve done in the past, we are going to take a measured approach to build up confidence in our safety assessments before further expanding our operations.
How do you deal with scenarios where the AV is in a line of cars and there is a bus/truck/large vehicle ahead that obscures the stop light? This is super common in San Francisco. What factors are in play for the AV to deduce that there is a red light ahead instead of an obstruction that it needs to go around? Is this fully map/rules-based, or does the vehicle take other clues into account to make a decision?
Here’s a few ways the Waymo Driver is well-positioned to navigate scenarios like this:
1. Our sensors are uniquely positioned to provide our system a comprehensive view of the world — whether it’s our peripheral vision system enabling us to peek around traffic or from our central perception dome’s added height.
2. Likewise, our maps can inform our planners and behavior systems to better inform us of what could be happening on the road and to inform our path based on deductive reasoning.
3. And lastly, when in doubt, the Waymo Driver will wait it out and look for other context clues, such as what other road users are doing, to inform its path of action.
Does simulation become more or less important to validating safety as the waymo driver matures?
Simulation plays an important role in our safety framework, and it is hard to imagine a point at which it is “less important.” For example, evaluating new software releases uses re-simulation of existing logs and scenario-based simulation testing. With that being said, our safety record from real-world, rider-only driving is very valuable for evaluating the Waymo Driver's performance. Rider-only operations provide high-fidelity, in-vivo safety signal, but I don’t think this necessarily devalues / replaces simulation - at least not for the foreseeable future. - John
How often do rides in San Francisco require intervention by a remote operator (e.g. per X passenger miles)? Has this metric improved over the past two years?
Hi, Trent here! Remote assistance is an industry standard part of full-stack AV operations, but the definition of ‘remote assistance’ varies from company to company. This is why I won’t be sharing numbers. But I want to provide more context on how it works at Waymo. We have a fleet response team that works remotely that we like to think of as air traffic control for our autonomously driven cars. If the Waymo Driver encounters a novel or ambiguous situation, it can request additional info. But it generally does not rely on this input from the fleet response, nor does it wait for that input. That is because most of these instances are resolved by the Waymo Driver without assistance, even before the fleet response team responds.
I believe Waymo helped draft IEEE P2846 standard for safety. Does the Waymo Driver have a set of safety rules, like Mobileye's Responsibility Sensitive Safety, to maintain safe longitudinal and lateral distance, avoiding cut-ins etc... Can you speak to how the Waymo Driver handles safety decisions?
That’s correct, we did co-author the 2846 standard, and it’s important to understand the role we give to that standard. 2846 establishes the notion of a “safety-related model” that is capable, through kinematic envelopes, of informing when the driving policy of an ADS may not be appropriate for the current road environment. The standard itself establishes the reasonably foreseeable assumptions that can be made about the behavior of other road users that serve as an input to such a model. Waymo uses these models for behavioral evaluation, but not as a direct rule-based generation of trajectories. In other words, we use them to evaluate how we behave, but not to determine HOW TO behave in the first place. - Trent and Francesca
Human drivers break rules of the road all the time. Does Waymo ever break any rules in order to get to their destination in a timely matter?
Great question- The Waymo Driver is designed to follow the rules of the road. Take speeding for example- it’s designed to respect the posted speed limit but can also respond to dynamic situations like decreasing its speed for construction or speeding up to safely execute a lane change. We actually
published a study this summer with regard to speeding if you'd like to learn more. - Trent
How does waymo plan to communicate to other road users. What are some of the things you plan on communicating? Will there be regulations on what needs to be communicated?
All this said, without the presence of a human driver, our autonomously driven vehicles do communicate their actions and intentions to other road users. Because our cars operate autonomously without a human in the driver’s seat, Waymo leverages its core autonomous driving technology to help fill those gaps and communicate what our cars do to other road users so they respond accordingly. The Waymo driver will (along with a bunch of other things) nudge, start slowly, use all available vehicle indicators, and honk to help communicate intent.
The Waymo Driver also communicates several messages using audio and visual cues. One of our newest signals visualizes on the Waymo dome display to the folks behind us that we are yielding to a pedestrian. We also communicate that a rider may be entering or exiting the vehicle to help prevent dooring collisions. - John
What kind of high severity crash avoidance scenarios is waymo’s platform capable of?
Since high severity crashes are rare, we rely on simulation (validated by a smaller number of test track experiments) to evaluate the Waymo Driver’s performance in a range of scenarios. We’ve talked about some of those simulation efforts in the past (
CAT,
Human Fatal Crash Reconstructions). In addition to evaluating the types of events with human-driven vehicles we’ve observed through research and in our >40M miles of TO and >10M miles in RO driving, we also evaluate things that are unique to an engineered system (like the vehicle platform, sensors, etc.). We talk a little bit about this “V&V” type of activity in our
readiness determination paper. - Kris
How can the Waymo Driver tell if it gets a flat tire or there are other issues with the car itself?
The vehicle has various self-monitoring capabilities for handling a number of vehicle-related issues that might occur. In particular, Waymo vehicles have tire pressure monitoring and responsive action capabilities should a flat tire occur. - John
Full AMA:
patents.justia.com
