U.S. safety agency says 'did not assess' Tesla Autopilot effectiveness

[Az_Rael]

U.S. safety agency says 'did not assess' Tesla Autopilot effectiveness

Following that fatality, Tesla pointed to a 2017 NHTSA report on a May 2016 fatality involving a driver using Autopilot. The report said crash rates fell by 40 percent after installation of Autopilot’s Autosteer function, and noted that this number was derived from Tesla airbag deployment data.

The agency said on Wednesday its crash rate comparison “did not evaluate whether Autosteer was engaged” and “did not assess the effectiveness of this technology.”

Hope we get more info from Tesla on this subject.

 

[Az_Rael]

For reference, here is what the preliminary report stated:

https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF

5.4 Crash rates. ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to (21) and after Autopilot installation.(22) Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation.

21 Approximately one-third of the subject vehicles accumulated mileage prior to Autopilot installation.
22 The crash rates are for all miles travelled before and after Autopilot installation and are not limited to actual Autopilot use.

 

[mongo]

It would make sense that they are looking a time point of before/after not specific feature usage. So it may lump together the full AP feature suite, not just AP the auto-steering specific feature.

Active lane assist may be better than just lane departure warning, which is pretty impressive on its own:
Lane departure warning cuts crashes

Results of the new study indicate that lane departure warning lowers rates of single-vehicle, sideswipe and head-on crashes of all severities by 11 percent and lowers the rates of injury crashes of the same types by 21 percent. That means that if all passenger vehicles had been equipped with lane departure warning, nearly 85,000 police-reported crashes and more than 55,000 injuries would have been prevented in 2015.

I look forward to Tesla's new quarterly safety statistics report.

 

[Barklikeadog]

They keep throwing out this generic 40% number... serious accidents everywhere have been declining due to new safety features. Other vehicles without AP... but
with AEB, lane departure, and blind spot detection have seen large drops in recent years.

 

[croman]

There is no contradiction between the report and what NHSTA is saying. They are just clarifying that just having Autosteer installed reduced the crash rate during the studied timeframe by 40% (as compared to the Tesla vehicles that didn't pay for AP). The study couldn't and didn't tract whether using AP actually helps, just having the feature (and that is interesting because all Teslas from 2014-2016 had the safety features, Autosteer and others (summon) were the convenience portion of the software package).

 

[hacer]

They keep throwing out this generic 40% number... serious accidents everywhere have been declining due to new safety features. Other vehicles without AP... but
with AEB, lane departure, and blind spot detection have seen large drops in recent years.

Actually that is false. They haven't dropped at all, and certainly haven't had large drops.

According to the CDC's vehicle injury database for the USA it's been pretty much flat from 2013-2016, approximately the period over which the Tesla data was analyzed. Source: WISQARS (Web-based Injury Statistics Query and Reporting System)|Injury Center|CDC (you can generate your own queries to verify my spreadsheet).

Very long term (largely due to prevalence of air bags) fatalities have experienced large drops but there has been no drop in serious collisions in the USA recently.

You keep barking, but it's just noise.

 

[S4WRXTTCS]

I think it's important to distinguish two things.

The overall safety record of a vehicle that has available Autosteer, and the safety of Autosteer while under use.

Back when the fatality accident happened in the Florida everyone was wondering if AutoSteer significantly contributed to the crash. There were massive threads on here, and elsewhere of people debating it.

As part of the NHTSA investigation they looked into whether the availability of autosteer was causing an increase in crashes. In their comparison they found a 40% crash reduction AFTER the Autosteer feature was released.

What that meant to me is a normal safe driver wasn't any less safe in a car with available autosteer. Like they didn't suddenly start crashing into barriers, trucks, etc. at an increased rate.

Now keep in mind I wasn't too happy with the report because it didn't indicate what firmware version the data collection started at. It needed to start at V6.2 to include AEB, and some improvements to other systems along with TACC. It wasn't as good of an study as I would have liked. It did go a long ways into quashing fears that Autosteer made the cars less safe statistically, but brought up a lot of questions.

Then Tesla started assuming that the cars were 40% safer while IN autopilot or at least giving that impression, and this is what the NHTSA is trying to combat.

I find it mindboggling that the NHTSA didn't look that 40% reduction further. That level reduction is not just counter intuitive, but no one can explain how in the world that would be.

My assumption is the NHTSA screwed up, and that 40% includes other things that shouldn't be there.

 

[Barklikeadog]

Actually that is false. They haven't dropped at all, and certainly haven't had large drops.

According to the CDC's vehicle injury database for the USA it's been pretty much flat from 2013-2016, approximately the period over which the Tesla data was analyzed. Source: WISQARS (Web-based Injury Statistics Query and Reporting System)|Injury Center|CDC (you can generate your own queries to verify my spreadsheet).
View attachment 299292

Very long term (largely due to prevalence of air bags) fatalities have experienced large drops but there has been no drop in serious collisions in the USA recently.

You keep barking, but it's just noise.

and these stats are specifically tied to automobiles with lane assist, blind spot detect and automatic emergency braking???

 

[animorph]

I took NHTSA's concern to be that they did not officially and independently study Tesla's AP statistics. They only accepted the data as-is from Tesla and calculated that 40% number from the data as part of a previous investigation. By saying Tesla's "first iteration of Autopilot was found by the U.S. government to reduce crash rates by as much as 40%", Tesla was giving the 40% number more credibility (and government endorsement) than it deserved. Any other details are extraneous.

 

[hacer]

and these stats are specifically tied to automobiles with lane assist, blind spot detect and automatic emergency braking???

No. Is that what you were talking about? I certainly didn't read your post that way.

If that's what you meant, then any data has the same exact problem that you complain about Tesla: they have no way of knowing whether these features are turned on or off, only that the cars are equipped with the features. So I assumed you weren't applying a double standard. Anyway all the reports I've seen that attempt to examine cars equiped with those features use police-reported accidents, 72% of which are property damage only (thus not usually "serious accidents"). Tesla's numbers are for air-bag deployments, which usually result in some sort of injury for some party, but in the case of a single-car accident in a Tesla, they are so safe that there are often no injuries with air bag deployment. For the one study that I know of that looked at injuries, there was an 11% decrease when cars were equiped with FCW, and AEB, but it was barely statistically significant, and the results for lane departure were not statistically significant.

Please point to a study that looked at air-bag deployments or injuries so that we're talking about serious accidents and showed a statistically significant large reduction in accident rates.