FSD v12.x (end to end AI)

[spacecoin]

The chess analogy is encouraging. For years people wondered if a computer would ever be able to beat a human at chess. Many people said no.

Currently, no one can beat the best computers at chess.

Chess? Please... That's a trivial sandboxed small problem space that you can brute force and that doesn't require real-time decision making and isn't safety critical.

 

[powertoold]

We should all take a breather and stop to appreciate the immense effort and engineering that went into getting FSD, especially V12, into production. V12 doesn't just represent some NN training cluster, but years of infrastructure software buildout to test, curate, and iterate on videos from the fleet.

Despite the issues, it's a miracle that we have V12 in customers' hands.

If you want to see what the competition is like, watch this short clip, and this is with HD maps, whereas V12 is essentially driving through everything the first time:

 

[sleepydoc]

That's an interesting question, actually, since that mirror folding behaviour is something that basically zero humans will do, especially in a Tesla where you have to go hunting through menus to find the option. I've always thought it was a pointless behaviour (the car on V11 seems to have nowhere near enough confidence to go for any gaps where the difference in folded/unfolded mirrors is going to matter), but it's also a very easy one to do if you're programmatically shaping the driving behaviour so I can see why they did it.

With v12 supposedly trained on real behaviour presumably there would be hardly any training material of this for the NN to imitate, so examples of it still doing it would be interesting.

Beyond the actual folding of the mirrors, doing so only buys you 4, maybe 6 inches, tops. This is behavior no human routinely does so where are they getting material to train the NN? If the car has to go that close I wouldn’t trust Tesla Vision to be accurate enough anyway.

 

[sleepydoc]

@AlanSubie4Life ,

Can you man up and explain why you give a "laugh" emoji for every post I make? Pretty lame behavior.

It’s not just you. They’ve got his Prozac dose up so high he smiles and laughs at everything!

 

[FSDtester#1]

It’s not just you. They’ve got his Prozac dose up so high he smiles and laughs at everything!

When you say they, don't you mean you prescribed him too much Doc?

 

[azred]

Too bad Elon's not. He might let Chuck get it early at some point but Elon can be petty and unforgiving.

Are there any non-CA based YouTube stars with v12? The guy in Michigan that probably posts more YouTube “adventures with Tesla” than almost anyone still doesn’t have it either.

 

[FSDtester#1]

Are there any non-CA based YouTube stars with v12? The guy in Michigan that probably posts more YouTube “adventures with Tesla” than almost anyone still doesn’t have it either.

You are talking about @DirtyT3sla and he is a TMC member.
I don't know of anyone outside CA worth watching with V12.

 

[sleepydoc]

Are there any non-CA based YouTube stars with v12? The guy in Michigan that probably posts more YouTube “adventures with Tesla” than almost anyone still doesn’t have it either.

it appears that most/all of the V12 downloads have gone to cars in CA. The presumption is that they started with a small set of 'local' testers first, identified some issues and halted the roll out while they worked to fix them.

 

[sleepydoc]

When you say they, don't you mean you prescribed him too much Doc?

"I can neither confirm nor deny any prescriptions I may have written for @AlanSubie4Life"

 

[PianoAl]

Beyond the actual folding of the mirrors, doing so only buys you 4, maybe 6 inches, tops. This is behavior no human routinely does so where are they getting material to train the NN? If the car has to go that close I wouldn’t trust Tesla Vision to be accurate enough anyway.

This is what folding the mirrors is: Showing off.

Admit it, it looks really cool when it gets to a tight spot and casually folds in its mirrors as if sentient. Who doesn't go, "Whoa!"?

 

[Daniel in SD]

The chess analogy is encouraging. For years people wondered if a computer would ever be able to beat a human at chess. Many people said no.

Currently, no one can beat the best computers at chess.

I’m not sure why it’s encouraging. It took way longer for computers to beat humans at chess than many people predicted!

In 1968, Levy and artificial intelligence (AI) pioneer John McCarthy were at a party hosted by Donald Michie. McCarthy invited Levy to play a game of chess which Levy won. McCarthy responded that 'you might be able to beat me, but within 10 years there will be a computer program that can beat you.' Levy suggested they bet on it, and Michie agreed to up the ante. Other AI experts signed on later, with the bet total reaching £1,250.[22][23][24][25] In 1973, Levy wrote:[26]

Clearly, I shall win my ... bet in 1978, and I would still win if the period were to be extended for another ten years. Prompted by the lack of conceptual progress over more than two decades, I am tempted to speculate that a computer program will not gain the title of International Master before the turn of the century and that the idea of an electronic world champion belongs only in the pages of a science fiction book.

Kasparov didn’t lose a match until 1997!

 

[tmoz]

How do they tweak a NN release? Add a bunch of miles/incidents of special drives featuring how to do a certain maneuver correctly or how to behave in certain situations and then retrain the whole thing again? Or can they retrain certain subsets? I'm not an expert on NN, so I'm asking for info from those of you who are knowledgeable in the field.

 

[Supcom]

How do they tweak a NN release? Add a bunch of miles/incidents of special drives featuring how to do a certain maneuver correctly or how to behave in certain situations and then retrain the whole thing again? Or can they retrain certain subsets? I'm not an expert on NN, so I'm asking for info from those of you who are knowledgeable in the field.

I believe that you feed it additional videos of the desired behavior. Starting over from scratch is very expensive so you would like to avoid that

 

[JB47394]

I believe that you feed it additional videos of the desired behavior. Starting over from scratch is very expensive so you would like to avoid that

Does the math work out to allow modular training? That is, can I do all my stop sign training on one network, my right turn training on another, identical network, my speed keeping training on another, etc, ad nauseum, then perform some math to merge them together?

 

[vamike]

I feel that end to end will never be able to handle enough edge cases to achieve one severe collision every few million miles (current human driven Tesla performance).

I could easily see e2e improving on human driving because I'm not sure how relevant edge case are. I admit I don't have data to back this, but my gut says the majority of human injury/fatal crashes are not situational "edge cases" but instead routine driving scenarios caused by inattentiveness, distraction, falling asleep, poor driving skills, etc.. e2e will improve routine driving dramatically, while edge cases are by defintion so few that relavive e2e vs. human performance improvement or degradation may be statistically irrelevant.

 

[mongo]

Does the math work out to allow modular training? That is, can I do all my stop sign training on one network, my right turn training on another, identical network, my speed keeping training on another, etc, ad nauseum, then perform some math to merge them together?

If you mean merge as in training the sub actions on multiple identically sized networks and turning then into one same size network, no.
If you mean merge as in training the sub actions on multiple identically sized networks and then putting them in parallel followed by an additional network (with more training for that section), yes.

However, what is more likely is using a network of some size between those two options and training it on all data.

 

[Daniel in SD]

I could easily see e2e improving on human driving because I'm not sure how relevant edge case are. I admit I don't have data to back this, but my gut says the majority of human injury/fatal crashes are not situational "edge cases" but instead routine driving scenarios caused by inattentiveness, distraction, falling asleep, poor driving skills, etc.. e2e will improve routine driving dramatically, while edge cases are by defintion so few that relavive e2e vs. human performance improvement or degradation may be statistically irrelevant.

Yes, I'm sure the vast majority of crashes are not due to "edge cases." Humans are amazingly good at handling edge cases. But we're talking about achieving a serious collision every few million miles. How many situations will the e2e system encounter during that distance that are not just an interpolation of scenarios in the training data?
Of course I'm also just guessing based on my experience with ChatGPT-4.

 

[powertoold]

V12 in wide release will blow your minds

I believe the ChatGPT moment will be possible with V12

Btw, I'm still enjoying / preferring average auto-max

The most delightful thing about 12.2.1 is that it works better when there's more traffic

 

[sleepydoc]

I could easily see e2e improving on human driving because I'm not sure how relevant edge case are. I admit I don't have data to back this, but my gut says the majority of human injury/fatal crashes are not situational "edge cases" but instead routine driving scenarios caused by inattentiveness, distraction, falling asleep, poor driving skills, etc.. e2e will improve routine driving dramatically, while edge cases are by defintion so few that relavive e2e vs. human performance improvement or degradation may be statistically irrelevant.

Here’s the problem - if FSD handles non edge cases with 100% success but edge cases are only 20% there will still be a lot of FSD accidents and (justifiable) public outcry. It needs to handle both.

 

[AlanSubie4Life]

majority of human injury/fatal crashes are not situational "edge cases" but instead routine driving scenarios caused by inattentiveness, distraction, falling asleep

It's easy to see that if FSD fixed 95% of these problems and eliminated those accidents (say it eliminated 95% of the 80% of accidents due to such issues), and everyone used FSD, the overall accident rate could easily be far higher than without FSD. It depends on the competence of FSD (which would not need to be very high to fix 95% of these problems!).

e2e will improve routine driving dramatically,

Fixing the problems you've listed won't necessarily improve routine driving dramatically. These are "easy" problems to fix.

But the overall quality of driving has to be really high! Completely setting aside edge cases, FSD can't be causing accidents that wouldn't have happened before. Imagine a world where we had no drunk, inattentive, or sleepy (etc.) drivers! Think of how much lower the accident rate would be. FSD needs to be very close to as good as that level of human performance just to break even.

Of course, if only the really bad drivers were allowed to use FSD, the situation could be different.

Unfortunately this is all a theoretical thought experiment, since no data on FSD performance is available to the public. It's certainly possible for end-to-end FSD to improve overall safety (at some distant future point). We'll probably never know though, unless legislation is passed, which it presumably won't be. Even if accident rates come down, we won't know why.