FSD Beta v11.x

The next big milestone for FSD is 11. It is a significant upgrade and fundamental changes to several parts of the FSD stack including totally new way to train the perception NN.

From AI day and Lex Fridman interview we have a good sense of what might be included.

- Object permanence both temporal and spatial
- Moving from “bag of points” to objects in NN
- Creating a 3D vector representation of the environment all in NN
- Planner optimization using NN / Monte Carlo Tree Search (MCTS)
- Change from processed images to “photon count” / raw image
- Change from single image perception to surround video
- Merging of city, highway and parking lot stacks a.k.a. Single Stack

Lex Fridman Interview of Elon. Starting with FSD related topics.

Here is a detailed explanation of Beta 11 in "layman's language" by James Douma, interview done after Lex Podcast.

Here is the AI Day explanation by in 4 parts.

Deep Understanding Tesla FSD Part 1: HydraNet

From Theory to Reality, Analyze the Evolution of Tesla Full Self-Driving

saneryee-studio.medium.com

Here is a useful blog post asking a few questions to Tesla about AI day. The useful part comes in comparison of Tesla's methods with Waymo and others (detailed papers linked).

Twelve Questions about Tesla Autopilot Technologies Exposed at 2021 AI Day

On Aug. 19 2021, Telsa held an AI day , exposing more technologies about Autopilot , from perception, planning, data labeling, simulation…

yuhuang-63908.medium.com

 

[JulienW]

I don't think that's entirely correct--- where do you think the planner gets stop sign data from for signs NOT picked up by vision? And the driving computer has been requesting map data for a while now that gets fed back to it...

Yes, overlooked this. But the MCU is not in any way making and sending driving decisions to the FSD computer. So in effect Mapped Stop Sign info is likely sent to the FSD Computer and when it starts to stop the FSD Computer sends a Stopping at a Stop sign command to the MCU to be rendered.

 

[enemji]

Just to add I suspect the MCU visualization rendering engine is not fully compatible with v12. This may be why we are missing so much stuff. If the case it will probably need a rewrite to add back all the missing parts but that is likely a low priority now.

Or it could be that v12 is not complete and just not sending all this info yet. Whatever the reason we are missing a LOT visualization info now.

Labels are missing. Everything it sees needs to be matched to a object library

 

[MP3Mike]

Everything it sees needs to be matched to a object library

Not for FSDb V12, it doesn't work that way. The actual driving portion has no idea what anything is, and doesn't care. It just knows when it sees something that looks like "this" that you should do "that."

 

[JulienW]

Not for FSDb V12, it doesn't work that way. The actual driving portion has no idea what anything is, and doesn't care. It just knows when it sees something that looks like "this" that you should do "that."

AKA: Occupancy Network flushed down the toilet.

 

[MP3Mike]

AKA: Occupancy Network flushed down the toilet.

No, the Occupancy Network is still used for Park Assist and the visualizations.

 

[cyborgLIS]

Is the occupancy network not the car's 3D mapping of the environment around it, derived from the camera feeds? How can we have the visualization without that? Or is it still present but no longer used by v12?

 

[enemji]

Not for FSDb V12, it doesn't work that way. The actual driving portion has no idea what anything is, and doesn't care. It just knows when it sees something that looks like "this" that you should do "that."

I am talking about the visualization engine.

 

[MP3Mike]

I am talking about the visualization engine.

Yep, and they may have had to dumb down the V11 visualization engine so that the V12 driving engine has enough CPU resources available.

 

[JulienW]

No, the Occupancy Network is still used for Park Assist and the visualizations.

Meant v12 is flushing Occupancy Network down the toilet. Besides Park Assist we still have V11 on highways using it. Of course all of them will be replaced by NN V12 type code in "two weeks" or "by the end of the year", which ever comes LAST.

 

[JHCCAZ]

AKA: Occupancy Network flushed down the toilet.

The occupancy network is still there In the weights and will be refined through training, even if no longer in the explicit code.

Besides, occupancy is a prerequisite for flushing.

 

[Hiline]

Just re-read the first post of this thread. So we’ve been in V11 for 2 years now. Wow that’s a long time.

 

[JulienW]

Just re-read the first post of this thread. So we’ve been in V11 for 2 years now. Wow that’s a long time.

No that is where it was announced. Most got it starting around a year ago.

 

[sleepydoc]

No that is where it was announced. Most got it starting around a year ago.

And there was a fair amount of evolution during that time (until they completely abandoned v11 to work on v12)

 

[Hiline]

Ah so v11 was announced, then a year later it was rolled out, and then another year later it was replaced. Seems like v12 has had a shorter announcement-to-rollout time gap. Hopefully v12 also gets replaced by a non-Beta v13 in less than a year...

 

[Bladerskb]

Ah so v11 was announced, then a year later it was rolled out, and then another year later it was replaced. Seems like v12 has had a shorter announcement-to-rollout time gap. Hopefully v12 also gets replaced by a non-Beta v13 in less than a year...

FSD v8 - 2020
FSD v9 - 2021
FSD v10 - 2022
FSD v11 - 2023
FSD v12 - 2024
FSD v13 - 2025

 

[MarcG]

FSD v8 - 2020
FSD v9 - 2021
FSD v10 - 2022
FSD v11 - 2023
FSD v12 - 2024
FSD v13 - 2025

Almost. I got FSD Beta 10.2 on October 15 2021.

 

[rlsd]

FSD v8 - 2020
FSD v9 - 2021
FSD v10 - 2022
FSD v11 - 2023
FSD v12 - 2024
FSD v13 - 2025

There will be no v13. v13 will be skipped to become v14. 13 is a bad luck number and it will drive the customers away.