FSD v12.x (end to end AI)

[mongo]

I'd agree with usually should-- for super obvious things like the left that no longer exists I mentioned for example-- or road closes signs (which AFAIK are still pretty hit and miss)... but not all-- see below-





OTOH, we've also seen cases (posted, not personally) where the map had a stop sign that does exist but is overgrown or blocked to the point the camera might miss it. Should it ignore the map and drive right through it?

Much like all the local differences in laws about everything from red-light turns to parking rules, It's one of many problems that there's just no easy/obvious solution for.... (speed limits are another tricky one where the "if it doesn't see a sign" behavior has varied from time to time and place to place sometimes using map data, sometimes not)

Yeah
Kids stole the stop sign
Or, more often, power is out and traffic signals are off, in the countryside, in an area you don't know well

 

[SOULPEDL]

Should it ignore the map and drive right through it?

Schrödinger's Stop sign, I call it. Only stops when others are looking.

Yes it drives right through it (sometimes). Here's my map lol. (Not tested in the last few months.)
|
_______>Stop _____
| trees
^
me
on FSD

The setup is tricky bc the stop sign is block by trees, less than 50 ft after the right turn. So it's literally straightening out the vehicle after the last turn before the next stop sign... leaving no set up time and not even sure the front or pillar camera have it in view until right at the intersection. But it will just roll right through like it's in California or something.

However, if there are ANY other vehicles at this intersection, it ALWAYS stops because of the other cars waiting (3-way stop).

It's aware when others are looking and behaves, (which is how I drive, sometimes). But a cop would never see it run this intersection. It's like Schrödinger's Stop Sign, OK if others are looking. This is a feature.

 

[lzolman]

Schroeder's Stop sign, I call it. Only stops when others are looking.

Schrödinger's. That other guy plays piano for Peanuts.

 

[SOULPEDL]

Schrödinger's. That other guy plays piano for Peanuts.

Fixed it thx… I still cant spell it though.

 

[Usain]

So most of you I see here also post in the investment so you've already seen how I think robotaxi will play out. But for those who haven't here it is:

1. For Tesla to start a robotaxi service, they will pick a city where FSD V12 already performs extremely well.
2. Then Tesla will gather lots of extra data from that city to make FSD even better in that locale.
3. Then comes testing and launch in that city.
4. Tesla then demonstrates that not only is it capable of launching a robotaxi service, but it does so at a much lower cost than Waymo (and Uber!).
5. Rinse and repeat in other cities.
6. Meanwhile, the FSD team also continues to add more data from all around and FSD continues to get better at the driving task in general.

(side note: Does everyone agree that #1 is intuitively obvious? Tesla can't start a robotaxi service everywhere at once.)

I believe that Tesla can accomplish the launch in only 1 to 3 years. We have apparently seen how using more data from California seems to make the system better in California than in other places. So using extra data from one city to launch a robotaxi service years sooner makes sense.

@Knightshade has argued that using the extra data for the target city will cause too many regressions that would keep FSD from working well everywhere else.

I believe that Tesla engineers would be able to curate the data properly so regression would be minimal and they would quickly fix any actual regressions that come along. I don't see it as being much different from the way they create and test any other new version.

 

[Daniel in SD]

So most of you I see here also post in the investment so you've already seen how I think robotaxi will play out. But for those who haven't here it is:

1. For Tesla to start a robotaxi service, they will pick a city where FSD V12 already performs extremely well.
2. Then Tesla will gather lots of extra data from that city to make FSD even better in that locale.
3. Then comes testing and launch in that city.
4. Tesla then demonstrates that not only is it capable of launching a robotaxi service, but it does so at a much lower cost than Waymo (and Uber!).
5. Rinse and repeat in other cities.
6. Meanwhile, the FSD team also continues to add more data from all around and FSD continues to get better at the driving task in general.

(side note: Does everyone agree that #1 is intuitively obvious? Tesla can't start a robotaxi service everywhere at once.)

I believe that Tesla can accomplish the launch in only 1 to 3 years. We have apparently seen how using more data from California seems to make the system better in California than in other places. So using extra data from one city to launch a robotaxi service years sooner makes sense.

@Knightshade has argued that using the extra data for the target city will cause too many regressions that would keep FSD from working well everywhere else.

I believe that Tesla engineers would be able to curate the data properly so regression would be minimal and they would quickly fix any actual regressions that come along. I don't see it as being much different from the way they create and test any other new version.

Can you quantify “extremely well”?

 

[Knightshade]

@Knightshade has argued that using the extra data for the target city will cause too many regressions that would keep FSD from working well everywhere else.

No, I have not.

I see moving this to the proper section hasn't changed your habit of ignoring the actual posts you're replying to and instead building strawmen to knock down.


Below are the relevant things the people in the discussion actually said

once a behavior is "learned" by the AI/NNs for a given geo-fenced area, we don't know the impact of unlearning it or re-learning it when more areas w/ diff rules are added.

In reply you posted this-

That's not the way it works. Learning is cumulative. Just because FSD is better in San Francisco does not make it worse somewhere else.

FSD V12 needs more training in rainy areas because it needs more training in rainy areas. Once that happens, V12 will get better at driving in the rain. But that won't make it worse in San Francisco.

If you oversample in a geofenced area, the FSD will get really good at driving in that area. But that won't make it worse anywhere else.

This misunderstands a fair bit about how learning actually works and I corrected you by pointing out:

It might if people drive differently there.

This is one of the reasons driving is a really hard problem.
AI could get great at Go or Chess because every time you sit down the rules are the same. The knight always moves the same way, on every chess board, in every country.

Driving is much more locally different than that. The simplest example is it's legal to turn right on red some places and not others... heck it's legal to turn LEFT on red some places and not others. And those rules can vary not just by state, but even by city, or PART of a city. And there's often not signs making this super clear.

Without hardcoding that's a tough thing to "solve" for a general driving solution. And there's lots of other examples more subtle (differences in road markings, types of intersections, restricted lanes, signage, etc).

To be clear I don't think it's an unsolvable problem, but I DO think it means overfitting to one area CAN make it worse in others.

You then kept going down the rabbit hole ignoring everyone pointing out you seemed to have a poor understanding of the system and how NN training worked, and increasingly making up things nobody said and then insisting those imaginary arguments were wrong.

To the point you accused me of claiming end to end could never work, despite the fact I never said anything remote like that, and as you can see from my first post explicitly said the issue I raised was solvable--- I was only correcting your mistaken claim the issue did not exist....and telling me I should sell all my stock since I didn't think FSD could be solved with NNs (which, again, I said the opposite of)

I believe that Tesla engineers would be able to curate the data properly so regression would be minimal

I guess this is progress from your original claim, quoted above, that there would not be any regression because "that's not how learning works" or something.

But it's still wrong.

Regressions elsewhere are more likely to occur as you overtrain in a specific area, and especially when your ENTIRE system is NNs which are much harder to predict (or, without local testing everywhere even FIND) regressions.

Without explicit rules and code it gets harder not easier for the system to "learn" the right behavior across a wide array of locations that all have different rules. Especially if a vastly overfitted amount of your data comes from one specific location.


Again none of this is insoluble (geotagged training data, if the NNs are intended to consider that as an input, for example will help this, but you'll need data from a lot of DIFFERENT places, not a lot of it from one place.... you'll likewise need it to consider time/date, holidays, and other factors that all change rules of driving for humans but aren't obvious from a 30 second clip of video)

But your entire theory here seems to be "Tesla will just quickly make per-city perfectly safe NNs and everything will be great" based on.... I'm honestly not even sure what.

and they would quickly fix any actual regressions that come along. I don't see it as being much different from the way they create and test any other new version.

The training methods for V12 are entirely different from every wide release version ever

So if you don't see it as much different from "any other" new version it again underlines how poorly you understand everything behind the scenes and how it all works.

 

[Usain]

Can you quantify “extremely well”?

By "extremely well" I mean in the 99th percentile compared to FSD performance in other cities.

I'm just saying that it makes sense to start in one of the cities where FSD already works best.

 

[Usain]

No, I have not.

I see moving this to the proper section hasn't changed your habit of ignoring the actual posts you're replying to and instead building strawmen to knock down.


Below are the relevant things the people in the discussion actually said



In reply you posted this-



This misunderstands a fair bit about how learning actually works and I corrected you by pointing out:




You then kept going down the rabbit hole ignoring everyone pointing out you seemed to have a poor understanding of the system and how NN training worked, and increasingly making up things nobody said and then insisting those imaginary arguments were wrong.

To the point you accused me of claiming end to end could never work, despite the fact I never said anything remote like that, and as you can see from my first post explicitly said the issue I raised was solvable--- I was only correcting your mistaken claim the issue did not exist....and telling me I should sell all my stock since I didn't think FSD could be solved with NNs (which, again, I said the opposite of)






I guess this is progress from your original claim, quoted above, that there would not be any regression because "that's not how learning works" or something.

But it's still wrong.

Regressions elsewhere are more likely to occur as you overtrain in a specific area, and especially when your ENTIRE system is NNs which are much harder to predict (or, without local testing everywhere even FIND) regressions.

Without explicit rules and code it gets harder not easier for the system to "learn" the right behavior across a wide array of locations that all have different rules. Especially if a vastly overfitted amount of your data comes from one specific location.


Again none of this is insoluble (geotagged training data, if the NNs are intended to consider that as an input, for example will help this, but you'll need data from a lot of DIFFERENT places, not a lot of it from one place.... you'll likewise need it to consider time/date, holidays, and other factors that all change rules of driving for humans but aren't obvious from a 30 second clip of video)

But your entire theory here seems to be "Tesla will just quickly make per-city perfectly safe NNs and everything will be great" based on.... I'm honestly not even sure what.









and they would quickly fix any actual regressions that come along. I don't see it as being much different from the way they create and test any other new version.

If I misunderstood the meaning of your posts then I apologize. You have also misunderstood the meaning of mine.

Let's just stick to my latest post. I tried to be as clear as possible.

Which of my 6 points do you disagree with and why?

 

[mgs333]

So most of you I see here also post in the investment so you've already seen how I think robotaxi will play out. But for those who haven't here it is:

1. For Tesla to start a robotaxi service, they will pick a city where FSD V12 already performs extremely well.
2. Then Tesla will gather lots of extra data from that city to make FSD even better in that locale.
3. Then comes testing and launch in that city.
4. Tesla then demonstrates that not only is it capable of launching a robotaxi service, but it does so at a much lower cost than Waymo (and Uber!).
5. Rinse and repeat in other cities.
6. Meanwhile, the FSD team also continues to add more data from all around and FSD continues to get better at the driving task in general.

(side note: Does everyone agree that #1 is intuitively obvious? Tesla can't start a robotaxi service everywhere at once.)

I believe that Tesla can accomplish the launch in only 1 to 3 years. We have apparently seen how using more data from California seems to make the system better in California than in other places. So using extra data from one city to launch a robotaxi service years sooner makes sense.

@Knightshade has argued that using the extra data for the target city will cause too many regressions that would keep FSD from working well everywhere else.

I believe that Tesla engineers would be able to curate the data properly so regression would be minimal and they would quickly fix any actual regressions that come along. I don't see it as being much different from the way they create and test any other new version.

Do you really think Tesla robotaxis are anywhere remotely close to deployment? I saw some posts in the investment subforum in the past where people were expecting it to launch any day and now such posts are making their way into the FSD subforum.

 

[lzolman]

For an investor right now, that would be really nice but I'd be overjoyed with a V12 that moves the needle in the correct direction toward L3. Until we get L3, we're sure as hell not getting an autoaxi.

 

[Mardak]

1. For Tesla to start a robotaxi service, they will pick a city where FSD V12 already performs extremely well

Are you thinking of a Tesla operated service or that existing owners will be able to have their vehicles drive without a human potentially based on 12.x end-to-end? Tesla probably will monitor 12.x safety and disengagement metrics to know which areas have sufficient coverage that also are low risk, and those areas might not need special training as they were already fit and don't even need to do things that might then overfit.

Even looking at where 11.x already does better could be a proxy for areas that are "easier" such as those without multiple lanes to avoid needing to decide which to be in. This could point towards lower density areas that don't have complex traffic patterns, and this could also match up with under-served markets for taxis. Although it seems like 12.x should be much more comfortable and smoother than 11.x, presumably Tesla is improving safety that might be harder to assess and is the actual limiting factor for deploying.

 

[Daniel in SD]

If you look at actual robotaxis (Waymo and Cruise) you will see that none of their safety issues are location specific. They have to do with interactions with humans (drivers, cyclists, and pedestrians).
How are you going to use end-to-end training to solve the incident that got Cruise shut down?

 

[AlanSubie4Life]

How are you going to use end-to-end training to solve the incident that got Cruise shut down?

They won’t because that is not their goal with current hardware! Not sure why anyone thinks that robotaxis might happen with that.

It would be great to get L3 on some surface streets, but even that is an impossibility of course. Seems like it should be possible, but it’s actually fantastically hard.

Fake it ‘til you make it is the name of the game. A VERY long term game.

Fortunately nobody is really that close on robotaxis, at least based on the Waymo snippets I have watched.

 

[drtimhill]

This doesn't track. They are obviously training individual areas as we see test drivers testing Chuck's UPL.

It's way too early to say "they did it". We are still seeing level 2 ADAS that still makes mistakes on curated drives. I don't see anything that doesn't suggest robotaxi is imminent. TBH, it appears to be another small step from 11.4.9 comparing his videos, but hopefully the backend will be easier to make substantial changes as the progress with FSD has been incredibly slow.

They were TESTING Chuck's UPL, not necessarily training on it. But the meta point remains, V11 has lots of special case code for these cases, whereas V12 has one unified approach. Sure, you still have to include UPLs in the training data, but that turns up not as a result of cherry picking the data, but more as a result of the sheer volume of that dataset. Now, two be sure, you can have bias in that dataset; if Tesla use data gathered from the fleet they are going to get a LOT of data for the US west coast (with its large population os Teslas) and rather less for (say) the rust belt, and that does have the potential to impact the cars abilities outside of its comfort areas.

 

[diplomat33]

So most of you I see here also post in the investment so you've already seen how I think robotaxi will play out. But for those who haven't here it is:

1. For Tesla to start a robotaxi service, they will pick a city where FSD V12 already performs extremely well.
2. Then Tesla will gather lots of extra data from that city to make FSD even better in that locale.
3. Then comes testing and launch in that city.
4. Tesla then demonstrates that not only is it capable of launching a robotaxi service, but it does so at a much lower cost than Waymo (and Uber!).
5. Rinse and repeat in other cities.
6. Meanwhile, the FSD team also continues to add more data from all around and FSD continues to get better at the driving task in general.

(side note: Does everyone agree that #1 is intuitively obvious? Tesla can't start a robotaxi service everywhere at once.)

I believe that Tesla can accomplish the launch in only 1 to 3 years. We have apparently seen how using more data from California seems to make the system better in California than in other places. So using extra data from one city to launch a robotaxi service years sooner makes sense.

@Knightshade has argued that using the extra data for the target city will cause too many regressions that would keep FSD from working well everywhere else.

I believe that Tesla engineers would be able to curate the data properly so regression would be minimal and they would quickly fix any actual regressions that come along. I don't see it as being much different from the way they create and test any other new version.

This approach would go against everything Elon seems to believe in. He has been very clear that he is against geofencing or training FSD to be specialized in an area. Rather, Elon has said that he wants FSD to be generalized to work everywhere. So I doubt that Tesla will start curating data to specialize FSD to be a robotaxi in a specific area. Rather, Tesla will continue to gather data everywhere until FSD no longer requires supervision anywhere. It would actually be easier for Tesla to do what you suggest. But that is how Elon wants to do things and I don't see things changing unless somehow Elon did leave Tesla.

 

[Usain]

Do you really think Tesla robotaxis are anywhere remotely close to deployment? I saw some posts in the investment subforum in the past where people were expecting it to launch any day and now such posts are making their way into the FSD subforum.

I think within 1 to 3 years, end-to-end (V12+) will be ready for a robotaxi trail. I have never believed that until V12. With some oversampling of a target city, I think it will perform better than Waymo. So, yes.

 

[Usain]

Are you thinking of a Tesla operated service or that existing owners will be able to have their vehicles drive without a human potentially based on 12.x end-to-end? Tesla probably will monitor 12.x safety and disengagement metrics to know which areas have sufficient coverage that also are low risk, and those areas might not need special training as they were already fit and don't even need to do things that might then overfit.

Even looking at where 11.x already does better could be a proxy for areas that are "easier" such as those without multiple lanes to avoid needing to decide which to be in. This could point towards lower density areas that don't have complex traffic patterns, and this could also match up with under-served markets for taxis. Although it seems like 12.x should be much more comfortable and smoother than 11.x, presumably Tesla is improving safety that might be harder to assess and is the actual limiting factor for deploying.

I am thinking of a Tesla operated service. It's possible that Tesla would partner with Uber or Lyft, but Tesla has consistently indicated that they would monetize the service. But that doesn't mean the existing owners would not be able participate in the service. Tesla has also indicated that existing owners would have a chance to cash in as well.

I agree with your point that it's possible that V12 already works so well in some area that overfit is not needed. If so, great!!!

But as soon as Tesla picks a city for the rollout, they will want FSD to perform as well as possible in that city. So it follows that overfitting would help to accomplish that goal.

 

[Usain]

If you look at actual robotaxis (Waymo and Cruise) you will see that none of their safety issues are location specific. They have to do with interactions with humans (drivers, cyclists, and pedestrians).
How are you going to use end-to-end training to solve the incident that got Cruise shut down?

In the Cruise situation, a pedestrian was hit and thrown into the path of the Cruise vehicle. I don't think there is any way to solve that.

I don't actually fault Cruise that much for doing the wrong thing after the pedestrian was run over. The Cruise vehicle tried to pull over, which did make the situation worse. But a human driver might also do the wrong thing in such a situation.

Cruise got in trouble because they allegedly tried to hide what actually happened.

How would you use end-to-end training to make FSD do a better job than Cruise did? Given that Tesla probably doesn't have a lot of clips of pedestrians getting thrown under their cars, Tesla would need to rely on simulations. And indeed Tesla is doing that.

 

[Usain]

Fortunately nobody is really that close on robotaxis, at least based on the Waymo snippets I have watched.

Robotaxi is already a reality with Waymo. They have driverless vehicles that pick up passengers and take them to their destination.

The problem with Waymo is that, from what I can tell, they are nowhere near profitability. I believe that Tesla is taking a low-cost approach that will allow them to operate a robotaxi service and make a lot of money from it.