FSD v12.x (end to end AI)

[kpanda17]

Anyone get the spring update? FSD ver inside?

 

[kpanda17]

I believe the Collision Avoidance warning (at least what is sent to the MCU) is independent of the action FSDS takes.

Seems so, but should it be linked?

 

[JulienW]

Seems so, but should it be linked?

Pretty sure anything that is sent to the MCU is post FSD computer and is just driver FYI. So it is a static one way post information link only.

 

[Ben W]

This. In my daily driving I’ve reached the point where the majority of my interventions and disengagements are preference issues, not safety issues. With the significant exception of speed. 12.5 still can’t manage to pick the right speed so I’m continually goosing the accelerator to get it up to speed. If Tesla could fix that I would barely need to do anything.

I'd say the majority of my disengagements are correctness issues: the car gets into an incorrect lane (e.g. trying to go straight from a left-turn-only lane), or misses a turn entirely (most U-turns), or stops at a red light badly positioned in the lane, preventing drivers from getting past to make a right turn, and I get honked at. Potholes and road debris avoidance are also significant causes; these might cause damage to the car if not to the occupants. And agreed on needing to goose the speed constantly.

What I also see a lot are cases where the system gives up or is heavily degraded due to sub-optimal visibility: rain, fog, nighttime with very low road lighting, sun glare, dirt on glass. These are cases where radar/lidar could supplement the cameras to allow the car to continue driving safely. "L4 but only in good weather" is a non-starter, because weather and conditions can unexpectedly change mid-drive. This is why I think the current 8-camera vision-only sensor suite is inadequate for L4/Robotaxi. Not because it can't do it in perfect conditions, but because perfect conditions are never guaranteed.

The other blocker for accepted-by-society L4 is whether the system is respectful enough of surrounding cars and drivers. Currently it does something every mile or two that is not immediately dangerous or illegal, but could seriously annoy nearby drivers. (And this doesn't even count the too-slow speed.) (And road rage is a danger.) This may be solvable by more training data or by a faster computer / deeper network, but it is emphatically not solved yet.

 

[kpanda17]

I'd say the majority of my disengagements are correctness issues: the car gets into an incorrect lane (e.g. trying to go straight from a left-turn-only lane), or misses a turn entirely (most U-turns), or stops at a red light badly positioned in the lane, preventing drivers from getting past to make a right turn, and I get honked at. Potholes and road debris avoidance are also significant causes; these might cause damage to the car if not to the occupants. And agreed on needing to goose the speed constantly.

What I also see a lot are cases where the system gives up or is heavily degraded due to sub-optimal visibility: rain, fog, nighttime with very low road lighting, sun glare, dirt on glass. These are cases where radar/lidar could supplement the cameras to allow the car to continue driving safely. "L4 but only in good weather" is a non-starter, because weather and conditions can unexpectedly change mid-drive. This is why I think the current 8-camera vision-only sensor suite is inadequate for L4/Robotaxi. Not because it can't do it in perfect conditions, but because perfect conditions are never guaranteed.

The other blocker for accepted-by-society L4 is whether the system is respectful enough of surrounding cars and drivers. Currently it does something every mile or two that is not immediately dangerous or illegal, but could seriously annoy nearby drivers. (And this doesn't even count the too-slow speed.) (And road rage is a danger.) This may be solvable by more training data or by a faster computer / deeper network, but it is emphatically not solved yet.

Following you on this. Are we saying Waymo works during a torrential downpour?
Think there will be cases where any automation won’t work, autos, humanoids, robots, etc
Gotta like good old humans, we are fully weatherproof all the way to a weather death threat.

 

[sleepydoc]

I'd say the majority of my disengagements are correctness issues: the car gets into an incorrect lane (e.g. trying to go straight from a left-turn-only lane), or misses a turn entirely (most U-turns), or stops at a red light badly positioned in the lane, preventing drivers from getting past to make a right turn, and I get honked at. Potholes and road debris avoidance are also significant causes; these might cause damage to the car if not to the occupants. And agreed on needing to goose the speed constantly.

What I also see a lot are cases where the system gives up or is heavily degraded due to sub-optimal visibility: rain, fog, nighttime with very low road lighting, sun glare, dirt on glass. These are cases where radar/lidar could supplement the cameras to allow the car to continue driving safely. "L4 but only in good weather" is a non-starter, because weather and conditions can unexpectedly change mid-drive. This is why I think the current 8-camera vision-only sensor suite is inadequate for L4/Robotaxi. Not because it can't do it in perfect conditions, but because perfect conditions are never guaranteed.

The other blocker for accepted-by-society L4 is whether the system is respectful enough of surrounding cars and drivers. Currently it does something every mile or two that is not immediately dangerous or illegal, but could seriously annoy nearby drivers. (And this doesn't even count the too-slow speed.) (And road rage is a danger.) This may be solvable by more training data or by a faster computer / deeper network, but it is emphatically not solved yet.

Following you on this. Are we saying Waymo works during a torrential downpour?
Think there will be cases where any automation won’t work, autos, humanoids, robots, etc
Gotta like good old humans, we are fully weatherproof all the way to a weather death threat.

1. I have no expectation of FSD working in a downpour. Just like a new driver, you learn to drive in good conditions first, then go from there.
2. @Ben W - Just how do you expect the system to drive If the cameras are obscured? Do you not clear the snow off the windshield and windows before driving?
3. I do still experience some lane selection issues but they are vastly improved from 11.4
4. Humans aren’t fully weatherproof - actually the opposite. We need to be inside a weatherproofed car with the heat and wipers on! We do have a greater ability to accommodate suboptimal conditions, though.

 

[JB47394]

Shouldn’t the collision avoidance detection communicate with FSD and cause it to slow down/stop?

Shouldn't the collision detection warning system be taking its cues from the far more advanced driver assist system? If the driver assist is comfortable with dealing with the situation, why alert the driver? If it's not comfortable, then it should be sounding an alarm, regardless of the reason.

I'd certainly like to see a scale on the screen that indicates FSD's confidence level. Or a background shading system that moves through a spectrum of color.

 

[JulienW]

Shouldn't the collision detection warning system be taking its cues from the far more advanced driver assist system? If the driver assist is comfortable with dealing with the situation, why alert the driver? If it's not comfortable, then it should be sounding an alarm, regardless of the reason.

I'd certainly like to see a scale on the screen that indicates FSD's confidence level. Or a background shading system that moves through a spectrum of color.

I believe the driver assist code is through the FSD Computer too. Not sure how you determined it is "far more advanced" than FSDS code. Do you have a source to site or just speculation? If FSDS is active then it is in full control. You can't have 2 different programs fighting over control. If using FSDS you can turn ACW to Minimum to lessen the warnings.

Also I bet being a NN/AI system FSDS is always confident even when wishy washy between lanes.

 

[JulienW]

Just for kicks what is next?

1. 12.3.7/24.3.x (where I would put my on the lowest point)
2. 12.3.7/24.9.x
3. 12.3.7/24.15.x (Spring update before ???, not likely for us)
4. 12.4.x/24.3.x
5. 12.4.x/24.9.x
6. 12.4.x/24.15.x (WILD a$$ dreaming thinking Tesla would do it all)

 

[j0shm1lls]

Unfortunately those didn't exist when I did it lol, it was like 24 years ago or something close to that.

 

[cyborgLIS]

Just for kicks what is next?

1. 12.3.7/24.3.x (where I would put my on the lowest point)
2. 12.3.7/24.9.x
3. 12.3.7/24.15.x (Spring update before ???, not likely for us)
4. 12.4.x/24.3.x
5. 12.4.x/24.9.x
6. 12.4.x/24.15.x (WILD a$$ dreaming thinking Tesla would do it all)

I can't wait to get off the 12.3 path. 12.3.5 and 12.3.6 have been net regressions for me. Slightly less frequent lane wobble than previous 12.3s but worse lane selection and somehow even more timid at intersections. Completely unusable at any remotely busy intersection without accelerator interventions. Also just forgets to put up turn signals sometimes.

Earlier today it gave up its spot in the left lane just a few hundred feet from an upcoming turn. When it couldn't get back into the left lane, it tried to make the left from the right lane. Not cool.

 

[lzolman]

The other blocker for accepted-by-society L4 is whether the system is respectful enough of surrounding cars and drivers. Currently it does something every mile or two that is not immediately dangerous or illegal, but could seriously annoy nearby drivers. (And this doesn't even count the too-slow speed.) (And road rage is a danger.) This may be solvable by more training data or by a faster computer / deeper network, but it is emphatically not solved yet.

Lately just about everything I get annoyed about relates to it doing something "not fast enough". Obviously stop signs, but now I'm seeing this reluctance to make a lane change when I ask for it. If there are no cars in the lane I want to change into, it's immediate. But if there's a vehicle, even waaaaay back so an immediate lane change would be very safe, it demurs until the opportunity is basically lost. I have to either disengage to make the lane change or turn off the blinker and have people rolling their eyes at me (as I'd then be doing myself).

 

[MP3Mike]

But if there's a vehicle, even waaaaay back so an immediate lane change would be very safe, it demurs until the opportunity is basically lost. I have to either disengage to make the lane change or turn off the blinker and have people rolling their eyes at me (as I'd then be doing myself).

In my experience if you are within about a mile of the next turn it will ignore your command to change lanes, and just cancel the signal.

 

[fasteddie7]

Lately just about everything I get annoyed about relates to it doing something "not fast enough". Obviously stop signs, but now I'm seeing this reluctance to make a lane change when I ask for it. If there are no cars in the lane I want to change into, it's immediate. But if there's a vehicle, even waaaaay back so an immediate lane change would be very safe, it demurs until the opportunity is basically lost. I have to either disengage to make the lane change or turn off the blinker and have people rolling their eyes at me (as I'd then be doing myself).

I have to press and hold the turn signal button to force it to lane change, otherwise it turns off the signal and ignores my request. Not sure how you would do it in a vehicle with a stalk.

 

[rlsd]

I forgive 12.3.6 stupid lane changing.

https://twitter.com/x/status/1786877421205241883

 

[kabin]

Edge case, not disengaged
2023 MYP with USS running 12.3.6
Multi lane busy road, car ahead makes a bad slow left, while passing it in the same lane, the slowness of the turning vehicle and my approaching MYP, my collision avoidance warning sound goes off, the MYP goes slightly to the right to avoid and passes

Question
Shouldn’t the collision avoidance detection communicate with FSD and cause it to slow down/stop?

I did not engage, but consider it an edge case.

Yes if it was a capable integrated system. But it only covers the frontal path area. Of course the driver can dial it down to reduce needless alerts.

On the other hand, recent versions of FSD feel slightly dialed down to avoid false alarms and put more reliance on the supervisor.

 

[JB47394]

Not sure how you would do it in a vehicle with a stalk.

Hold down/up the stalk.

 

[cyborgLIS]

Speaking of the forward collision alert, how reliable do you guys feel it is? I find it gives alerts when there is no actual danger, and fails to give alerts in situations that actually require prompt action. I would expect more from a car as tech oriented as a Tesla (and with such high safety ratings). Do other cars' collision warnings perform this way? I'm especially concerned about automatic emergency braking activating at the appropriate time, since I expect it to be dependent upon the forward collision detection.

 

[Ben W]

1. I have no expectation of FSD working in a downpour. Just like a new driver, you learn to drive in good conditions first, then go from there.
2. @Ben W - Just how do you expect the system to drive If the cameras are obscured? Do you not clear the snow off the windshield and windows before driving?
3. I do still experience some lane selection issues but they are vastly improved from 11.4
4. Humans aren’t fully weatherproof - actually the opposite. We need to be inside a weatherproofed car with the heat and wipers on! We do have a greater ability to accommodate suboptimal conditions, though.

1. The issue is not that it can't drive in a downpour today. The issue is that the HW3/4 sensor suite (eight fixed cameras pressed up against glass) may be fundamentally incapable of overcoming the resulting degree of visual distortion and occlusion. Clearly we can't expect a car to ever drive in an F5 tornado, but adding more and different/better sensors may expand the potential ODD from ~95% uptime (down 5% due to poor weather) to ~99.99%, which will be necessary for Robotaxi.
2. Humans do it by sitting far back from the windshield, and moving our heads around, which allows us to see acceptably well through even downpour-obstructed window glass. Fixed-position cameras pressed up against glass (particularly side/rear cameras without wipers) can't do this. Radar and lidar could overcome this to some degree, at least enough to have the car's performance degrade gracefully instead of suddenly. Or Tesla could add something more exotic like mini-wipers or laser cleaning, but this presumably wouldn't be retrofittable to HW3/HW4.
3. It's possible you're right regarding lane selection, but I haven't noticed much improvement with v12, at least on my daily routes.
4. See #2. For a personally-owned L4 car that encounters extreme conditions and goes into "safe mode", a human can manually take over and keep driving. But if an unpiloted Robotaxi encounters such conditions, it is completely stuck. (Until or unless conditions improve.) If getting the Robotaxi unstuck requires human intervention, such as if the car encounters a pigeon with particularly good aim, or the aforementioned snow accumulation, it may be stuck for quite a while.

 

[Ben W]

Following you on this. Are we saying Waymo works during a torrential downpour?
Think there will be cases where any automation won’t work, autos, humanoids, robots, etc
Gotta like good old humans, we are fully weatherproof all the way to a weather death threat.

A car with Waymo-level capabilities would either continue working or degrade gracefully (go into safe mode, pull over, and temporarily stop) during a torrential downpour. A Tesla tends to panic and fail abruptly. Clearly no car can drive in a Cat 5 hurricane. The difference is 99.99% uptime vs 95% uptime, and the manner in which the car handles the extreme conditions when encountered. For Robotaxi, the car will have to fail gracefully enough that it can safely pull itself over and stop. Red flashing "System Error: Take Control Immediately" is not an option.