Welcome to Tesla Motors Club
Discuss Tesla's Model S, Model 3, Model X, Model Y, Cybertruck, Roadster and More.
Register
This site may earn commission on affiliate links.
Doesn't back propagation have the effect of turning any involved layers into a monolith? I'm seeing this as a case of a 'perception' network being nothing more than a starting point for the overall system's training. That is, after its initial training, nothing is going to require that the perception network deliver stop signs and lane lines. Those signals will be tweaked and tuned by the back propagation until the ultimate output - control - is optimized.

Tesla could

1) Do everything from scratch in a totally new architecture

2) Do an end-to-end architecture that incorporates some of the previous module architectures, but retrain the weights from scratch

3) Do an end-to-end that incorporates some of the previous module architectures, and initializes training using their initial weights.

My guess is they are doing #3 at least wrt to the perception stack. They will lop the core layers of that thing into their V12 architecture. They will probably cut of a few of the last layers (which are usually used for the final classification / regression outputs) so they *won't* have explicit output or backprop on things like object detection, segmentation, kinematics, etc... but they will start with the same weights for the layers that do the heavy lifting of perception.

When they train, they can choose to allow those weights to still be updated, or they can freeze them if they so choose. I have no idea what they are doing now, but ultimately I would think you would want to unfreeze them to allow the model to find more robust modeling of all the nuances found in real data.

So working up to V12 via V9, V10, and V11 is not a lost cause. Some of those models can be incorporated into V12 and in weight-initialization, which can dramatically speed up training time. And if not that, they are still useful for training debugging outputs like visualizations.
 
When they train, they can choose to allow those weights to still be updated, or they can freeze them if they so choose. I have no idea what they are doing now, but ultimately I would think you would want to unfreeze them to allow the model to find more robust modeling of all the nuances found in real data.
Yeah, that's what I was thinking about freezing/unfreezing to start from existing models:
Indeed, adding the control neural network to the end of that to reach end-to-end doesn't need autolabeled clips as you suggest because freezing perception to train control converts the raw video to intermediate outputs/inputs during the forward pass, and then including the control weights to compare with the expected output allows backpropagation, e.g., to the beginning of control. This also allows for training the entire stack with nothing frozen although it'll be a balancing act of making sure other outputs, e.g., for visualization, aren't broken.
Curious, do you have experience / thoughts on how complex it is to balance training schedules that freeze/unfreeze certain parts? Presumably doing cycles of training just perception then just control then both (then other heads?) and repeat could shift the weights to get out of local maximums even at the same learning rate?
 
  • Like
Reactions: RabidYak
If I block 44, will the pounding never stop and they won’t give me 38.10?
Has anyone on 27 been moved to 38?
Would be interesting to see if those with 11.4.7/8 get 12 before rhe 11.4.4 group
If 27.x goes to 38.9, then those currently on 38.9 or less block 44 and wait and get early access to 12
What a mess
You will never get 2023.38.10. No cars owned by ordinary users will get this version. None. It's a test version that was sent to some employee cars to expand the testing of V12. Tesla is likely a long way from releasing V12 to us.

I am on the development branch (2023.27.7). Other than a few employee cars, none of us on this branch have been sent any newer software. If we were, it would be a major discussion on these forums.

The best-case scenario, which is almost certainly NOT to happen, is for Tesla to release V12 as part of the Holiday update. If so, then you could expect everyone on the production and development to receive it at about the same time.

Most likely, V12 will start to go to the OG testers sometime in the first quarter next year with rollout to the development branch around April (my guess) and finally to the production branch a month or so after that. That's assuming that Tesla continues the two branch software track. In any event, Tesla is likely at least five releases away from rolling out to everyone. And it will likely require a month between releases for assessing telemetry, retraining and quality testing each release.
 
Perhaps I missed it, but I didn't hear any mention of FSD v12 at the cybertruck delivery event. Seems like if v12 was imminent, Elon would have touted it for CT.

To be honest, the launch event was very light on details in general. If Elon had mentioned the V2H or the optional battery pack range extender at the event people would have thought it had gone better.

So I'm not surprised he didn't mention FSD at the event.
 
  • Like
Reactions: pilotSteve
I could be wrong but my understanding is that end-to-end still has perception. After all, you can't do autonomous driving without some sort of perception. The architecture of the perception is just different. End-to-end does not do the old style perception where humans manually label objects and you have an object detection NN module that draws 3D boxes around objects. Instead, end-to-end embeds the perception with the planning. So it "understands" that a group of pixels is a pedestrian or a car and knows from training how to respond when it sees that grouping of pixels behave in a certain way.
I very much doubt they are doing this. In theory you could feed pixels in at one end and wire the NN up to the car controls At the other. Then you train the heck out of it. The trouble with this approach is training such a big network is very very expensive. Also, the car would not provide visualizations for the humans (not good for an L3 system). Probably Tesla are retaining the existing networks and then feeding those into the next network along with GPS etc data and THAT is driving the car.
 
  • Helpful
Reactions: pilotSteve
I very much doubt they are doing this. In theory you could feed pixels in at one end and wire the NN up to the car controls At the other. Then you train the heck out of it. The trouble with this approach is training such a big network is very very expensive. Also, the car would not provide visualizations for the humans (not good for an L3 system). Probably Tesla are retaining the existing networks and then feeding those into the next network along with GPS etc data and THAT is driving the car.

Exactly. There is a huge difference between end-to-end neural network inference (going forward), which is feasible, vs end-to-end neural network *training*, i.e. backprop exclusively from observed supervised controls signals from human drivers.

I suspect the engineers have to fuzz out this difference to Elon to keep him satisfied and out of their hair

I suspect there is still tons of perceptual supervision using the existing pipeline and datasets, and then on top a more neural control system---which might be a distillation of a classical off-line optimization based and rule based system, i.e. examples from the offline algorithm are used to train the approximating network which is used in inference on board. The goal here is if the good off-line optimization can't be done computationally efficiently on-board in time, but a neural approximation can.
 
To be honest, the launch event was very light on details in general. If Elon had mentioned the V2H or the optional battery pack range extender at the event people would have thought it had gone better.

So I'm not surprised he didn't mention FSD at the event.
Completely agree. When are people going to realize that vehicle unveilings are always short on details.
 
Teslascope has a pretty good track record.
This was interesting

TeslaScope V12 Post.png