Tesla autopilot HW3

[Daniel in SD]

What???

MobilEye doesn’t even have a system with the functionality of Nav on Autopilot on the road.

Pretty easy to have a “perfect record”. Yay, we’re the best. Zero accidents over zero miles!

Yes, Tesla’s lane keeping and auto lane changing is probably the best (though GM’s super cruise is arguably better in one way since it’s hands free).
I’m taking about all the other ADAS features like automatic emergency braking, pedestrian crash avoidance, etc.
Arguably those features are just as important to showing progress towards autonomous vehicles.

 

[kbM3]

Yes, Tesla’s lane keeping and auto lane changing is probably the best (though GM’s super cruise is arguably better in one way since it’s hands free).
I’m taking about all the other ADAS features like automatic emergency braking, pedestrian crash avoidance, etc.
Arguably those features are just as important to showing progress towards autonomous vehicles.

Tesla’s published stats indicate 2x miles / accident with Autopilot.

Please present stats of others you claim are better.

 

[rnortman]

Tesla’s published stats indicate 2x miles / accident with Autopilot.

Please present stats of others you claim are better.

Ever read "how to lie with statistics"? The problem is that miles driven on Autopilot are different miles than miles driven not on Autopilot. Unless they randomize their fleet to randomly disable Autopilot when an owner would otherwise want to use it, they really can't show causation with these sorts of statistics.

Come to think of it... maybe that explains the BS I've been experiencing with Autopilot recently....

 

[Saghost]

Ever read "how to lie with statistics"? The problem is that miles driven on Autopilot are different miles than miles driven not on Autopilot. Unless they randomize their fleet to randomly disable Autopilot when an owner would otherwise want to use it, they really can't show causation with these sorts of statistics.

Come to think of it... maybe that explains the BS I've been experiencing with Autopilot recently....

There's certainly some truth to what you say - and figuring out how much effect that has is going to be very difficult.

It does not, however, constitute the requested statistical evidence that another manufacturer has a better system.

 

[Daniel in SD]

Tesla’s published stats indicate 2x miles / accident with Autopilot.

Please present stats of others you claim are better.

I'm pretty sure Tesla's statistics wouldn't even count hitting a pedestrian as a collision. I'm not claiming any one manufacturer is better, I'm just saying we don't know. How do you quantify how well automatic emergency braking is working in a fleet? Pedestrian crash avoidance mitigation?

 

[turnem]

I agree that personal driving style shouldn't be the benchmark, but right now it often behaves in ways that can be a nuisance to other drivers, the worst example being phantom braking (which I see with increased frequency recently, with the car apparently reacting more to other cars in neighboring lanes when it shouldn't).

Agree. Just today it slammed on the breaks for a car that was turning in front of me. Mind you the car was already out of the way when it decided to hit the breaks so it really screwed up twice - it applied the brakes too late AND it applied them when it wasn't even remotely necessary.

 

[ZeApelido]

You do know that EVERYBODY developing autonomous vehicles uses cameras and deep learning? Literally every company doing this has vision systems like Tesla's, except generally with better cameras and more powerful inference hardware. And I'm sorry, which "guru" says "Elon is right"? And about what exactly?

Wow I just opened up a tensorflow session and built a CNN to train on stop signs. Just formed ZeInc. Valued at $10 billion. Am I a competitor?

No. Because it ain't about the model (mostly), it's about the data (mostly). I don't have it. BMW doesn't have it. Mobileye probably doesn't have it.

Well yes, Mobileye's entire "claim to fame" when they started out was that they were able to do ADAS with just a cheap camera, which was later extended to multiple cameras. Quite possibly this is what made Elon, with Tesla being a Mobileye customer, believe in the approach. However, also note this passage in the article you linked:

"That dozen cameras, plus of course GPS and a couple decades' worth of learning from the company's various imaging-based driver assistance systems, is enough to make the car drive itself.
Except that it isn't. At least, it won't be when it comes to delivering the kind of on-road redundancy that Mobileye's engineers demand. And that's why Mobileye is now working on a lidar and radar-based solution."

As Lex Fridman said in a recent lecture, vision-only systems rely on the driver as failsafe, while vision+lidar systems use the lidar (and potentially HD maps) as failsafe. If true, this would imply that the vision-only system would never go beyond level 2 autonomy.

If all the cameras fail, or cameras plus FSD chip fail, not sure the backup system needs to be much more than "slow the car to stop quickly".

 

[kbM3]

I'm pretty sure Tesla's statistics wouldn't even count hitting a pedestrian as a collision. I'm not claiming any one manufacturer is better, I'm just saying we don't know. How do you quantify how well automatic emergency braking is working in a fleet? Pedestrian crash avoidance mitigation?

To sum up, your claim that others were better than Tesla has no basis in fact?

 

[kbM3]

Ever read "how to lie with statistics"? The problem is that miles driven on Autopilot are different miles than miles driven not on Autopilot. Unless they randomize their fleet to randomly disable Autopilot when an owner would otherwise want to use it, they really can't show causation with these sorts of statistics.

Come to think of it... maybe that explains the BS I've been experiencing with Autopilot recently....

Once again. Give any evidence that other automakers are better than Tesla.

That was the original claim made.

 

[Daniel in SD]

To sum up, your claim that others were better than Tesla has no basis in fact?

Are there statistics to back that up? I'm not convinced that Tesla's system has a better safety record than Mobileye's system.

I think you've got me mixed up with someone else. I never said that Tesla's system is worse. I'm just saying we don't have the data on how well different automakers ADAS systems work in the real world. I know the IIHS is developing tests but I'm a little bit skeptical on how well those correlate to real world performance.

 

[kbM3]

I think you've got me mixed up with someone else. I never said that Tesla's system is worse. I'm just saying we don't have the data on how well different automakers ADAS systems work in the real world. I know the IIHS is developing tests but I'm a little bit skeptical on how well those correlate to real world performance.

I never claimed Tesla’s system was safer, so what in the world was the point of your post?

I’m claiming Tesla’s has the most features. If you want to claim: “so what they’re not as safe” then back it up, otherwise once again, what was the point of your post?

 

[Daniel in SD]

I never claimed Tesla’s system was safer, so what in the world was the point of your post?

I’m claiming Tesla’s has the most features. If you want to claim: “so what they’re not as safe” then back it up, otherwise once again, what was the point of your post?

It sounds like we're in agreement. I guess I should have asked you to clarify what you meant by "best". I think of ADAS as primarily about safety and I haven't seen any information about which system is the best. Tesla's has the most user visible features. It's not clear that the number of features would necessarily make one system better than another. Other systems may have more safety "features", I don't know.

 

[Eno Deb]

If all the cameras fail, or cameras plus FSD chip fail, not sure the backup system needs to be much more than "slow the car to stop quickly".

That's of course an extremely simple case and probably not what Fridman was thinking of. But what about failures that cannot be detected that easily by the vision system? For example, if a neural network mistakenly classifies a stationary object as a shadow or similar? If you have a lidar, it will detect the object and override the vision system in this case. If not, you have to rely on driver supervision (which means you're stuck with level 2 autonomy).

 

[ZeApelido]

That's of course an extremely simple case and probably not what Fridman was thinking of. But what about failures that cannot be detected that easily by the vision system? For example, if a neural network mistakenly classifies a stationary object as a shadow or similar? If you have a lidar, it will detect the object and override the vision system in this case. If not, you have to rely on driver supervision (which means you're stuck with level 2 autonomy).

I mean, this example is at the crux of the argument of camera-only vs camera + lidar. If they are going to argue you only need camera, these are the sort of situations it will obviously have to be able to manage with high fidelity.

Whether or not it will be able to, I don't know. With enough data samples and compute power, I would guess eventually yes.

 

[CK_Stuggi]

Don't forget you can get 3D info both from stereo vision and SLAM. So it's not like vision will easily fail.
Also if said object is covered in dark cloth, LIDAR isn't going to catch it either...

 

[electronblue]

Don't forget you can get 3D info both from stereo vision and SLAM. So it's not like vision will easily fail.
Also if said object is covered in dark cloth, LIDAR isn't going to catch it either...

That’s one benefit of having triple redundancy: visioin, radar and Lidar. That narrows down the failure scenarios quite bit.

Tesla has no redundancy of any distance towards the sides eg at a 90 degree turn. No stereo vision there either.

 

[CarlK]

I mean, this example is at the crux of the argument of camera-only vs camera + lidar. If they are going to argue you only need camera, these are the sort of situations it will obviously have to be able to manage with high fidelity.

Whether or not it will be able to, I don't know. With enough data samples and compute power, I would guess eventually yes.

It's not camera vs. camera + Lidar. It's if you can have large data base to train the neural net. You can't long as you need to have a Lidar in the car. A well trained NN is the key above whatever sensors you have. It's not just for image recognition but you need that to do things like to decide what is the correct path and to predict what outside cars and pedestrians are about to do. So even the argument added Lidar does not do harm does not stand (until it's cheap enough to be equipped in all cars).

 

[EinSV]

I thought the video in this link courtesy of @SPadival was on point for every FSD thread on TMC:

S Padival on Twitter

 

[CarlK]

What???

MobilEye doesn’t even have a system with the functionality of Nav on Autopilot on the road.

Pretty easy to have a “perfect record”. Yay, we’re the best. Zero accidents over zero miles!

Some people are changing narrative to safety when Tesla started to pull ahead in functionality. You are right nothing beats safety of those disabled cars in the junk yard.

 

[CarlK]

I saw this posted by "Clara Smith" on Electrek comment section I thought was very informative. I can understand why there were non-believers but things are starting to make sense now.

*******************************************************************************************************
Google (later Waymo) began their venture into self-driving in 2009, well before deep learning neural nets had been put to practical use with the availability of powerful GPU chips. The big breakthrough came in 2012 when deep learning proved practical for voice and image recognition. By that time the Google research team had settled on a strategy that emphasized the use of high definition maps and lidar sensors for localization along with traditional AI software algorithms and expert systems to identify and predict the behavior of vehicles and people. Although Waymo today incorporates deep learning convolution neural networks in their system design, the dependence on high definition maps and complex sensor fusion continues. The reason is that deep learning, a subset of machine learning, is not fully embraced by Waymo is that it requires vast amount of data to train them for accuracy. Obtaining that data is very expensive and time consuming to train the NN to handle the edge cases that trip up autonomous systems. Even if Waymo acknowledges the power of deep learning to achieve full car autonomy, they are not in a position to change their development strategy, and, I suspect, they don't see a need to change anyway. Not everyone believes that deep learning will deliver on the ultimate goal of FSD at Level 5. Lex Fridman does.

If deep learning is the way to achieve the holy grail that is full autonomy, Tesla is uniquely positioned to make it happen. They parted ways with Mobileye in 2016 setting out on a path that emphasized deep learning to achieve FSD. It's one of the reasons for believing that Tesla is far ahead of other car manufacturers in software. It isn't just the hardware and software per se that makes autonomous driving work, but the data that goes into adjusting the weights of the inputs to the nodes of the neural networks. Reverse engineering those values would be extremely difficult, if not impossible.
*************************************************************************************************************