FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[jebinc]

Jeb, will they finally fix my front end vibration from 30 to 48 mph????

Data to support my reply from yesterday.

 

[LowlyOilBurner]

The more drives logged on 11.4.4, the worse they are. Routes where lane selection was fixed on 11.3.6/11.4.2/3 are broken again. These are routes I travel weekly, so I know them well, it’s screwing up where it was fine before.

The apprehension around other vehicles and VRU’s is worse. Cars not moving waiting to turn into the road I’m on are being displayed as light blue, and ego is slowing for them. Pedestrians across the street is causing slowdowns. These are not “threats” to ego, there is NO reason to be slowing down or swerving for something that is not a threat.

With that being said, it does not move over enough for pedestrians on rural roads. This evening, I passed a poor guy walking his dog down one of these roads, my car moved over maybe a couple of feet for the guy. He yelled to move over. I had to turn around and apologize to the guy, I felt bad.

It still paces cyclists sometimes instead of moving over for them and giving them enough room and passing them. The vru behaviors worked fine way back on 10.3. But it slows down for a pedestrian standing on the other side of an intersection? Ok?

It overreacts for larger vehicles or trucks with trailers when making turns, another stupid behavior that’s been going on for a while.

Highway performance is decent, just wish it would get out of the passing lane quicker, or as soon as it passes a slower vehicle instead of just hanging in the passing lane.

I’m only using it now where I know it will work, for the most part.

Post Script:
They did not fix behavior on narrow rural unmarked roads, ego still comes almost to a complete stop when a vehicle is approaching.

It appears HW4 cars are doing better, maybe it’s time to trade in.

 

[rlsd]

The more drives logged on 11.4.4, the worse they are. Routes where lane selection was fixed on 11.3.6/11.4.2/3 are broken again. These are routes I travel weekly, so I know them well, it’s screwing up where it was fine before.

The apprehension around other vehicles and VRU’s is worse. Cars not moving waiting to turn into the road I’m on are being displayed as light blue, and ego is slowing for them. Pedestrians across the street is causing slowdowns. These are not “threats” to ego, there is NO reason to be slowing down or swerving for something that is not a threat.

With that being said, it does not move over enough for pedestrians on rural roads. This evening, I passed a poor guy walking his dog down one of these roads, my car moved over maybe a couple of feet for the guy. He yelled to move over. I had to turn around and apologize to the guy, I felt bad.

It still paces cyclists sometimes instead of moving over for them and giving them enough room and passing them. The vru behaviors worked fine way back on 10.3. But it slows down for a pedestrian standing on the other side of an intersection? Ok?

It overreacts for larger vehicles or trucks with trailers when making turns, another stupid behavior that’s been going on for a while.

Highway performance is decent, just wish it would get out of the passing lane quicker, or as soon as it passes a slower vehicle instead of just hanging in the passing lane.

I’m only using it now where I know it will work, for the most part.

Post Script:
They did not fix behavior on narrow rural unmarked roads, ego still comes almost to a complete stop when a vehicle is approaching.

It appears HW4 cars are doing better, maybe it’s time to trade in.

"Highway performance is decent, just wish it would get out of the passing lane quicker, or as soon as it passes a slower vehicle instead of just hanging in the passing lane."

In California there is no passing lane. If the car goes slow then it should move to the right. FSD moves my car to the right of the leftmost lane and it annoys me a lot of time.
It's hard for FSD to meet the legal driving requirements of each state. Probably FSD should compromise: Keep the lane selected by the user if the lane is on the route to destination.

 

[LowlyOilBurner]

"Highway performance is decent, just wish it would get out of the passing lane quicker, or as soon as it passes a slower vehicle instead of just hanging in the passing lane."

In California there is no passing lane. If the car goes slow then it should move to the right. FSD moves my car to the right of the leftmost lane and it annoys me a lot of time.
It's hard for FSD to meet the legal driving requirements of each state. Probably FSD should compromise: Keep the lane selected by the user if the lane is on the route to destination.

It should be user configured. An option for “leave passing lane ASAP” or something along those lines.

 

[lascavarian]

I originally thought that the sunglasses trick was a bug, but now I'm leaning towards it being a feature that Tesla is evaluating with its employee testers.

Sounds like a bad idea and a bug. The system safety design is based on torque on the wheel. You have discovered a serious bug that defeats critical safety design and need to report it.

Until Tesla changes that, violating the safety design puts you and others at risk little different than strapping a weight on the wheel.

If this is to be a feature (almost certainly not) then Tesla needs to be clear about it.

Sorry to sound harsh but this likely a serious bug.

 

[mtndrew1]

This morning as my car was hitting the brakes for invisible obstacles, making unnecessary lane changes and bouncing around the lane I look to my right and see…

 

[jebinc]

This morning as my car was hitting the brakes for invisible obstacles, making unnecessary lane changes and bouncing around the lane I look to my right and see…

View attachment 955253

J

 

[AlanSubie4Life]

Not news I am sure and presumably already reported but 11.4.4 breaks FCW (when set one step back from most sensitive).

 

[rlsd]

This morning as my car was hitting the brakes for invisible obstacles, making unnecessary lane changes and bouncing around the lane I look to my right and see…

View attachment 955253

Did Drive Pilot disengage?

 

[aronth5]

Glad to see Tesla employees seem to testing 11.4.5 longer than recent new builds. Hopefully what we finally get isn't full of regression.

 

[Supcom]

Glad to see Tesla employees seem to testing 11.4.5 longer than recent new builds. Hopefully what we finally get isn't full of regression.

Or, perhaps it didn't take long to reject this build and start training 11.4.6.

 

[lascavarian]

There is an interesting YouTube by Forbes of Liquid Neural Networks from MIT as applied to autonomous driving vision and targeting for drones. With just a handful of neurons the results are interesting.

There seems to be the implication that the weights are not locked allowing for some flexibility that advantages “attention”. YMMV

 

[sleepydoc]

I think nothing new in the notes is best for "meaningful improvement", so they can tweak 4.4 to work as intended vs adding new stuff. Same reason 3.6 works best for most people but the new capabilities of 4.x (albeit perhaps needed for the long-term) broke the 3.6 stability for many

There is virtually never anything new in the release notes for point version upgrades. We’re usually left to figure out for ourselves what‘s chagned.

 

[sleepydoc]

You might recall that Tesla had previously programmed FSD beta not to slow down suddenly when it (rightly or wrongly) detected a speed limit change. I'm pretty sure that was intended to mitigate the kind of complaints described in several messages above.

However, that policy generated complaints from people who were worried about getting tickets, for example when entering a small town speed zone where the car would take a long time to work itself down to the lower speed limit.

Then the nail in the coffin came from the Feds i.e. NHTSA. One of the issues they made Tesla "fix", in the recall from a few months ago, was this leisurely slow-down policy. I recall some reports that Tesla argued about this but lost the argument. So, I'm guessing the rate of slowdown was mandated.

Obviously this is really bad when the new speed limit is incorrect based on faulty map data - it happens to me every day in a particular spot. But people here are saying that sometimes the slowdown matches the legal requirement, yet they don't like it because the prevailing traffic doesn't honor it. I think in that case, Tesla is stuck with the NHTSA mandated behavior.

The problem was FSD took over a half mile to slow down in some cases and would actually actively apply power to prevent slowing (i.e. when going up a hill.) Beyond that, the change in speed with FSDb was significantly different (and slower) than it was with TACC.

As others have pointed out, you’re required to be at the posted limit at the sign, not after so the proper/legal method is to slow down is to start slowing before the sign so you’ve reached the limit by the time you pass it.

 

[sleepydoc]

Not news I am sure and presumably already reported but 11.4.4 breaks FCW (when set one step back from most sensitive).

FCW?

Edit: Forward Collision Warning - got it. Hadn’t heard that. Where did you see it/how do you know?

 

[uscbucsfan]

FCW?

Forward Collision Warning

 

[maddogs]

The problem was FSD take over a half mile to slow down in some cases and would actually actively apply power to prevent slowing (i.e. when going up a hill.) Beyond that, the change in speed with FSDb was significantly different (and slower) than it was with TACC.

As others have pointed out, you’re required to be at the posted limit at the sign, not after so the proper/legal method is to slow down is to start slowing before the sign so you’ve reached the limit by the time you pass it.

The problem is not with knowing what legally should be done, it's the lack of acceptance that when other human drivers are involved, the legal requirement is not always the safest course of action.

 

[uscbucsfan]

The problem is not with knowing what legally should be done, it's the lack of acceptance that when other human drivers are involved, the legal requirement is not always the safest course of action.

One of the recalls was for FSD to change speeds quicker, which Tesla complied with.

Just like the rolling stop option, this isn't the first or last thing the NHTSA will push to make the experience worse.

V11.4 has made the switch to relying on vision and GPS over map data it appears. Elon said it was necessary in November and would be implemented in the next few months, but 11.4 tries to follow lanes which leads to entering turning lanes (dangerously), bus lanes, taking the wrong turn...despite the map/planner showing the correct route.

Also, A big problem I've noticed after a 911 mile trip is that 45mph signs on frontage roads are seen and then the car responds, despite obvious knowledge of interstate speeds.

These big changes make a huge regression for tester experience.

 

[maddogs]

One of the recalls was for FSD to change speeds quicker, which Tesla complied with.

Just like the rolling stop option, this isn't the first or last thing the NHTSA will push to make the experience worse.

Only way to influence this is to influence your representatives and senators. I doubt there are enough of us to make a dent though.

 

[uscbucsfan]

Only way to influence this is to influence your representatives and senators. I doubt there are enough of us to make a dent though.

There's no logical pathway to resolve something like that through contacting politicians.