FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[ChimpledPot]

If some of us og testers (you know who you are) don't get this round, we gonna be pissed

I was part of the OG 100 safety score and I still don't have anything. What a joke.

 

[Acisplat]

Looks like those of us still on 69 are going to be waiting a bit more per Elon. Head over to the tweets thread if you want to see that.

https://twitter.com/x/status/1640206499820478465

This is what Elon's "one more round of refinement was referring to", not that we would have to wait for 11.3.4

 

[Matias]

Here's a slick little scenario to evaluate FSD system latency for a special case emergency stop.

The UI renders the full visible stop sign in about 1/2 second. Note it could even be longer if the logic is able to distinguish a partially visible stop sign (we know it does!). Chuck said he had to apply the brakes but it also appears FSD initiated brake action before his leg moved. So FSD initiated brakes about 1 second after the full stop sign was visible.

The whole unmarked road drive is really bad.

 

[eli_]

I wonder if the voice feedback thing just does a speech-to-text translation or they're actually listening to my ridiculous mumbling as I try to concisely sum up whatever stupid thing just happened.

 

[JulienW]

https://twitter.com/x/status/1640206499820478465

This is what Elon's "one more round of refinement was referring to", not that we would have to wait for 11.3.4

So which is quicker? Waiting on 11.3.420 or waiting on the Neural Nets to be reprogramed?

My App just told me to Fxxx off and stop bothering it every 30 minutes.

 

[JohnB007]

Lane changes were already really smooth on 69 though…

If any of this has any truth to it (and the last part about not signaling when you signal, when it is inappropriate, would be pretty awesome), I’ll be sure to come back on here and sing its praises. I have my doubts that it is quite that good though.

And I have a hard time understanding why they would make it this good on the highway and leave things poor on surface streets.

Sounds sublime though.

Sometime maybe I will find out.

I use FSDb daily so I think I can detect small changes. I used to think 69 was smooth compare to the previous version. But I really like this version's smoothness. That is, except when it enters the exit lane...

I'm of course comparing highway drives though. City drive smoothness difference is much smaller, and this is also based on the same 400km I drive weekly, so who knows how it may perform outside these route.

EDIT : Oh also keep in mind that my impression is 69 on Chill vs 13.1.1 on mostly Assertive (as it's not as annoying on when it wants to change lanes now)

 

[SO16]

Chuck’s unprotected left turn is kind of disappointing. Goes too slow, waits in lane with oncoming traffic.

I received 11.3.3 last night and will try out V11 full stack for the first time today.

 

[VanFriscia]

If some of us og testers (you know who you are) don't get this round, we gonna be pissed

I'm one, and I got it yesterday morning. I really think the selection process is random at this point in time.

 

[JB47394]

Lane changes were already really smooth on 69 though…

Lane changes by NoA on highways were not. That's what he was referring to.

And I have a hard time understanding why they would make it this good on the highway and leave things poor on surface streets.

Slower pacing, longer transitions. It isn't nearly as challenging a control environment.

 

[JB47394]

I wonder if the voice feedback thing just does a speech-to-text translation or they're actually listening to my ridiculous mumbling as I try to concisely sum up whatever stupid thing just happened.

They've gotta be converting it to text so that they can search the resulting database, but they may also save the original audio in case the conversion produces nonsense. It would be nice if they suggested words that would ensure they find our clips when searching for a problem, but I suppose leaving it unstructured will tell them a bit about how we perceive the system.

 

[aronth5]

Lane changes were already really smooth on 69 though…

If any of this has any truth to it (and the last part about not signaling when you signal, when it is inappropriate, would be pretty awesome), I’ll be sure to come back on here and sing its praises. I have my doubts that it is quite that good though.

And I have a hard time understanding why they would make it this good on the highway and leave things poor on surface streets.

Why surprised that so many people are touting highway improvements? The highway stack was four years old and if you were going to see improvements that would be the logical place to see it. Highways are so much simpler than city streets anyway that regression testing would be much easier and that has been demonstrated quite clearly.

Now that we have single stack I suspect Tesla will begin to focus more on all the city street problems we've been complaining about for a long time.

 

[kriskros91]

I finally got 11.3.3 on one of our two cars. Usually our Model X gets it first but this time my Model Y got it. Excited to try it out today.

I bet in about 2-3 weeks once all the excitement of V11 dies down we will get updated to 2022.6.x branch or maybe 2022.10 branch.

Also should this thread be unstickied soon? We need another thread to look forward to and muse about.. V12 maybe?

 

[ljbad4life]

I was able to complete a long 400 mile round trip drive the same day. On the way down on the old autopilot and the way back with fsdb highway.

I see some improvements but I see a lot of.... "non-improvements" particularly around dangerous moves and illegal lane changes. Things that would destroy my "tesla driving score" of 99% and have me ticketed by police if I drove like that. Zipping between cars frequent erratic lane changes., passing on the right, etc. They also eliminated regular autopilot, that will keep the car in it's lane without active lane change.

The less frequent lane changes setting remedies a lot of this! Unfortunately it gets reset every time you drive the car. I am 100% responsible for this car, with that in mind, I'll wait for the next update to try it again.

 

[uscbucsfan]

Passing on the right is such an odd complaint. If someone is in the left lane going to slow and refuses to move, the car should absolutely pass on the right...if it couldn't handle this, it would just be stuck.

Passing on the right is akin to going 1 mph, while technically illegal, it's not something you will get pulled for without other factors, it's also how 99% of real drivers behave on the road.

 

[JohnB007]

I was able to complete a long 400 mile round trip drive the same day. On the way down on the old autopilot and the way back with fsdb highway.

I see some improvements but I see a lot of.... "non-improvements" particularly around dangerous moves and illegal lane changes. Things that would destroy my "tesla driving score" of 99% and have me ticketed by police if I drove like that. Zipping between cars frequent erratic lane changes., passing on the right, etc. They also eliminated regular autopilot, that will keep the car in it's lane without active lane change.

The less frequent lane changes setting remedies a lot of this! Unfortunately it gets reset every time you drive the car. I am 100% responsible for this car, with that in mind, I'll wait for the next update to try it again.

Which setting did you use? Also, was your speed based lane change on or off?

I'm trying to figure out which combo works best for my kind of trips...

 

[Scott Chapman]

So, I checked just now (9:15am) and it says no update, last updated yesterday 8:45am?

 

[ljbad4life]

Which setting did you use? Also, was your speed based lane change on or off?

I'm trying to figure out which combo works best for my kind of trips...

I played around with the assertive, chill. I don't recall seeing speed based lane changes as an option anymore (not in the pop up menu while driving) or follow distance, FSDB decides now. The only thing that calmed down the lane changes was less frequent lane changes.

 

[ljbad4life]

Passing on the right is such an odd complaint. If someone is in the left lane going to slow and refuses to move, the car should absolutely pass on the right...if it couldn't handle this, it would just be stuck.

Passing on the right is akin to going 1 mph, while technically illegal, it's not something you will get pulled for without other factors, it's also how 99% of real drivers behave on the road.

Rolling stops, running red lights are also technically illegal but it's gotten NHSTA attention and forced Tesla to remove it. It should not being doing anything illegal by design unless explicitly told to by the driver who is responsible (for any tickets or any liability that arises unless tesla wants to put it's money where it's mouth is and certify level 3) or to avoid an accident. Hell lets drive on sidewalks if there's no one there while we are at it because it's technically illegal, but it's not hurting anyone.

 

[JohnB007]

I played around with the assertive, chill. I don't recall seeing speed based lane changes as an option anymore (not in the pop up menu while driving) or follow distance, FSDB decides now. The only thing that calmed down the lane changes was less frequent lane changes.

Try left or right clicking the right dial. It's a toggle button on that little tab at the bottom left.

At least if it does what it says, it will lessen lane changes. When, I'm not sure yet.

 

[AlanSubie4Life]

Lane changes by NoA on highways were not.

I was talking about NoA on 69 which I think is what was being referred to. (I don’t think that is any different than NOA on main branch, otherwise someone would have mentioned it!)

Looking forward to comparison.

Why surprised that so many people are touting highway improvements?

Not what I said. I expect lots of improvement. Just would be a bit surprised if lane changing were sublime as described. Since it cannot be done on high speed surface streets in a logical manner.

I guess I’ll see.

Slower pacing, longer transitions. It isn't nearly as challenging a control environment.

Just seems like a 55mph surface street should be pretty similar. I have not seen any preternatural ability there. Though maybe I just need to let it try more?

Anyway, we’ll all be able to judge for ourselves soon enough. This will be one of the first things I report on - especially the part where if doesn’t signal when you signal - very curious about that. I guess it has already been in videos?
Specific reference observation:

I can signal right, and it moves back super smoothly. Uses the same logic - checks for space before signal gets turned on.