What are the chances of Autopilot 3.0?

[JeffK]

Even vision could use redundancy. They could add more cameras, for example.

There are a total of five cameras plus radar that point forward. Three of those cameras and the radar are pointed in the direct path of the vehicle. I'm not sure how much more redundancy it needs to be able to pull itself over to the side of the road safely if a sensor goes out.

 

[AnxietyRanger]

There are a total of five cameras plus radar that point forward. Three of those cameras and the radar are pointed in the direct path of the vehicle. I'm not sure how much more redundancy it needs to be able to pull itself over to the side of the road safely if a sensor goes out.

The car needs to see more than just forward to FSD...

They have a front radar, having rear radars could be useful too... but staying just with vision, I can see many scenarios where adding cameras are useful, e.g.:

1) All around 3D vision (needs two cameras spaced apart)

2) Redundancy when a sensor is blocked by dirt or snow (so it doesn't have to pull over so easily)

3) Nose camera(s) to cover this scenario:

4) Front corner cameras to avoid the large bumper-level blindspots of current AP2 - approximated black areas in image:

5) Camera monitoring the driver?

Just some initial thoughts, I'm sure more ideas are out there...

 

[JeffK]

The strollers scenario is covered by ultrasonics.

 

[AnxietyRanger]

The strollers scenario is covered by ultrasonics.

Perhaps it is, but vision would be better for making sure and for redundancy. Vision would also allow for detecing further away, fast approaching objects in that situation where ultrasonics are useless...

I don't know which compromises Tesla will eventually make with an AP3 - all manufacturers of course make some compromises with their setups and different design decisions - just offering some ideas where improvements could be fathomed...

 

[Aieukl378]

Well, I really wonder if there will be a market( and of course the market will dictate the advancement of AP) for further levels of A.P.? Personally, I'm totally satisfied with Ap1 as I love it on highway drives but don't want/need off the highway. When I upgrade my S I'll opt for AP but really could card less about FSD...I wonder how may others feel this way?

 

[aWalkingShadow]

Well, I really wonder if there will be a market( and of course the market will dictate the advancement of AP) for further levels of A.P.? Personally, I'm totally satisfied with Ap1 as I love it on highway drives but don't want/need off the highway. When I upgrade my S I'll opt for AP but really could card less about FSD...I wonder how may others feel this way?

Even if you personally don't care about Level 5 FSD, FSD is a big deal. If cars can completely drive themselves, the the ownership equation changes completely; since you could have large fleets of autonomous cars, ride sharing could become more prevalent.
Then, there's the fact that people who own transportation companies no longer need to pay their drivers, and millions of people would lose their jobs.

 

[Bebop]

Well, I really wonder if there will be a market( and of course the market will dictate the advancement of AP) for further levels of A.P.? Personally, I'm totally satisfied with Ap1 as I love it on highway drives but don't want/need off the highway. When I upgrade my S I'll opt for AP but really could card less about FSD...I wonder how may others feel this way?

That is why I was wondering just how far you can keep going. It's like trying to make an iPhone thinner and thinner. You can only make it so thin.

When it comes to advancement of AP from a hardware POV with AP3 and beyond, I imagine it's just adding higher definition cameras and etc. Since AP2 is already supposed to be fully autonomous. As well as computer upgrades to the car for sure. Just like how every year the MacBooks feature a new processor of some kind and higher specs.

 

[JeffK]

Just like how every year the MacBooks feature a new processor of some kind and higher specs.

One would think right? Stupid 16 GBs of RAM.. /rant

 

[Bladerskb]

The strollers scenario is covered by ultrasonics.

You can't be talking about the same ultrasonic that has caused hundreds of potential/actual parking accidents...

 

[JeffK]

You can't be talking about the same ultrasonic that has caused hundreds of potential/actual parking accidents...

Looks like it worked and didn't kill the kid. Plus the new ones are upgraded.

 

[AnxietyRanger]

The strollers scenario is covered by ultrasonics.

You can't be talking about the same ultrasonic that has caused hundreds of potential/actual parking accidents...

Looks like it worked and didn't kill the kid. Plus the new ones are upgraded.

Be that as it may, AP2 really is quite blind in this pictured scenario.

Even if the ultrasonics bounce, what does FSD do? All it knows is something is there, but it can not see it, identify it without a nose camera. It can not step out of the car and go take a look. Not only that, the nose camera could also see a fast-approaching motorcycle or a delivery truck approaching behind of where those strollers are... A nose camera certainly would be useful IMO, as I already discussed before AP2 was released.

So AP2 definitely has, even when no camera is blocked by dirt etc., vision blindsports around the nose bumper level (no cameras see around the nose on bumper level) and in the pictured scenario when front cameras are blocked by nearby side walls. What will it do if ultrasonics react to something but it can not see and check what it is?

Of course when confidence levels reach those of a human driver it can ignore certain ultrasonic bounces and just drive out real slow, but still this is clearly one area where improvements could be made in the sensor suite.

 

[JeffK]

Be that as it may, AP2 really is quite blind in this pictured scenario.

Even if the ultrasonics bounce, what does FSD do? All it knows is something is there, but it can not see it, identify it without a nose camera. It can not step out of the car and go take a look. Not only that, the nose camera could also see a fast-approaching motorcycle or a delivery truck approaching behind of where those strollers are... A nose camera certainly would be useful IMO, as I already discussed before AP2 was released.

So AP2 definitely has, even when no camera is blocked by dirt etc., vision blindsports around the nose bumper level (no cameras see around the nose on bumper level) and in the pictured scenario when front cameras are blocked by nearby side walls. What will it do if ultrasonics react to something but it can not see and check what it is?

Of course when confidence levels reach those of a human driver it can ignore certain ultrasonic bounces and just drive out real slow, but still this is clearly one area where improvements could be made in the sensor suite.

I'm not seeing those "blindspots" in the coverage map...

 

[Saghost]

Do you think something like AP 3.0 hardware would come out in the next 2-3 years? (based on timeline of AP1 and AP2.)

My thinking is..maybe? I mean I feel probably not for another 5-7 years if anything. If you release a 3.0 in the next 2 years then you would screw over I feel a bunch of Model 3 owners who haven't even gotten their car yet and are still in line.

Is there really any point for a 3.0 if the car is fully autonomous with 2.0? I mean I suppose to add even further refinement to the system. What would you add in a 3.0 release to make it substantially better than 2.0 Full Autonomy?

Xavier is waiting in the wings...

I think it's very likely that Tesla will migrate to NVidia's system on a chip for self driving before the end of the year - 60% more processing capability on 12% of the power, and presumably cheaper as well.

That would be an obvious time to add any additional sensors Tesla has decided would be helpful since last year.

The other obvious time for changes is right before or right after they get level 4 code working on the streets effectively - which may or may not be tied to the coast to coast drive they are hoping to do by the end of the year.

 

[croman]

There is no way they are changing AP2 for at least 2 years. M3 just rolled out with this sensor suite. NOTE: I don't view switching the ECU to be changing AP2. I view AP2 as the sensor suite not what processes/interprets that sensor information.

 

[JeffK]

Be that as it may, AP2 really is quite blind in this pictured scenario.

Even if the ultrasonics bounce, what does FSD do? All it knows is something is there, but it can not see it, identify it without a nose camera. It can not step out of the car and go take a look. Not only that, the nose camera could also see a fast-approaching motorcycle or a delivery truck approaching behind of where those strollers are... A nose camera certainly would be useful IMO, as I already discussed before AP2 was released.

So AP2 definitely has, even when no camera is blocked by dirt etc., vision blindsports around the nose bumper level (no cameras see around the nose on bumper level) and in the pictured scenario when front cameras are blocked by nearby side walls. What will it do if ultrasonics react to something but it can not see and check what it is?

Of course when confidence levels reach those of a human driver it can ignore certain ultrasonic bounces and just drive out real slow, but still this is clearly one area where improvements could be made in the sensor suite.

I also forgot to note that the particular image you are posting is from Volvo's parking assist.

 

[AnxietyRanger]

I also forgot to note that the particular image you are posting is from Volvo's parking assist.

Yes, it is showing Volvo's nose camera. What's your point?

 

[JeffK]

Yes, it is showing Volvo's nose camera. What's your point?

Point is it's not a blindspot in AP2.

 

[Saghost]

There is no way they are changing AP2 for at least 2 years. M3 just rolled out with this sensor suite. NOTE: I don't view switching the ECU to be changing AP2. I view AP2 as the sensor suite not what processes/interprets that sensor information.

How do you know what's on the M3 this morning?

We'll know a lot more this evening, of course, but if they added a couple small or invisible sensors to it, I'm not sure how we'd know.

The gentlemen that have been poking through code in the capabilities thread said that it currently has hooks in it to check if the car is a 3, and separately if it has hardware 2.5 installed...

 

[AnxietyRanger]

I'm not seeing those "blindspots" in the coverage map...

I'm not sure why this is so hard to explain? Your image does nothing to counter my points about visual blindspots in the nose as well as the blindpots created by blocked B pillar cameras?

AP2 has visual blindspots in certain close-by and urban environment scenarios, at least in following areas (as shown also in my previous message). These could be improved by future iterations of the sensor suite, by adding e.g. more cameras and/or cross-traffic radars in the nose and corners of the car...

1) Left and right side of nose, at bumper level. B pillar front-facing cameras can not see low enough.

2) Front of nose, at bumper level. Windshield cameras do not have the FoV to see what's immediately in front of the car there (and anyway, couldn't see through the bonnet).

3) Front vision is blocked towards the sides when windshield or B pillar FoV is blocked. In the below scenario AP2 is blind towards the sides. A nose camera would help with that part.

In all of these cases, the only redundancy are the ultrasonics, which are useless in cross-traffic detection due to low range and low speed and have various downsides that vision and radars do not.

I have marked with red the areas where a Tesla AP2 is vision blind in this scenario - it can not see there outside of ultrasonics. Ultrasonics might pick up the strollers, but not any faster cross-traffic, and most importantly no visual contigency for the AP2 to act on, if B pillar cameras are blinded by obstacles...

 

[AnxietyRanger]

Point is it's not a blindspot in AP2.

It is for vision and radar, yes it is - so basically all cross-traffic detection and anything that can't be picked up or identified sufficiently by ultrasonics, which are limited.