Perhaps Tesla will include the Chill/Average/Aggressive selector in the training set along with sample videos demonstrating the different behaviors.
-
Want to remove ads? Register an account and login to see fewer ads, and become a Supporting Member to remove almost all ads.
FSD v12.x (end to end AI)
- Thread starter Buckminster
- Start date
-
- Tags
- #fixphantombraking autopilot chill is not chill elon is an ass fix fsd extreme late deceleration fsd does not commit to go fsd does not maintain proper speed fsd hard and late braking fsd has decision wobble fsd jerky driving fsd late deceleration fsd sometimes may not detect train crossings fsd stops too early too quickly fsd stops too far behind line fsd tailgating in city streets
Yeah, it seems like Tesla is trying to address things in at least two ways given that they've continued to work on maps with 11.x even when knowing 12.x is coming in. Here's an example of what map data inaccuracies FSD Beta needs to deal with. OpenStreetMap indicates for the location of the Street View photo below that the right lane of 2 lanes merges left, and the subsequent segment for the intersection indicates there's 3 lanes for a U-turn on the left and right-turn-only on the right. Here's how it actually looks:
It seems like there's actually 4 lanes at this upcoming intersection with the right-turn-only lane forking a little bit sooner than the short U-turn fork and bay. (And the map data says upcoming undivided road is 2 lanes total when there's actually a center turn lane.) Strictly following some 11.x heuristics could miss out on the upcoming left bay and conservatively switch to the right lane which is totally unnecessary given it's going to merge just ahead anyway.
The Autopilot team could have written some special control logic to more accurately interpret the map data, but that can easily result in an unmaintainable mess of handling many possible other odd intersection data. Potentially with end-to-end, it could learn with enough training the various patterns of inaccurate map data to decide when it should switch lanes or not based on human examples of switching or not switching with a wide variety of accompanying map data. So hopefully 12.x will work fine with bad map data and do even better with correct map data.
I'm curious what does the TomTom map data show for that same spot? (I think Tesla gets more data from TomTom than they do from OpenStreetMap. Though I think parking lot data comes mainly from OpenStreetMap.)
I've had Autopilot suddenly disengage during light snow for a 2018 Model 3 while now with newer software, it happily drives in the same condition without a sudden large red "take over immediately" alert. This particular "fix" came from no longer aborting when the radar is covered by a thin layer of of snow and instead relying on Tesla Vision to determine lead vehicle position and speed. Conceptually 12.x will be another software update that could drive in conditions previous software would have failed, but perhaps being safe in heavy rain will be a more difficult problem to solve end-to-end than light snow.
How do you get TomTom Turn Restriction data? I can report that in addition to Speed Limit with their MapShare tool, but I'm not sure how to get the current data. I do see that the TomTom road connectivity data is different from OpenStreetMap and Google Maps, and it's more like Bing and Michelin Map as they might be based on TomTom. Here's a BBBike Map Compare of the intersection:
Notice how the left 2 maps change from divided to undivided just South of the McDonald's whereas the right 2 maps have that split at the top edge of the viewport corresponding to the end of the parking lot. TomTom matches the left maps, so this somewhat reflects the type of discrepancy end-to-end needs to deal with if different map providers don't even agree on how roads are connected. Will end-to-end be able to deal with not just incorrect map data but imprecise positioning of that data?
Neural networks can learn things in unexpected ways, so providing appropriate training data with desired motion profile could be tricky. Your example of "stop signs when no traffic is present" could be specially targeted, but perhaps the stopping behavior should also apply to other cases like when there's one other vehicle at the stop sign or at a red light or even stopping behind another vehicle. With traditional C++ code, it's more explicit that "this behavior" applies to "these situations" whereas neural networks might generalize or specialize in the wrong ways. For example, maybe it should actually stop with a different profile when the vehicle from the right arrives first, but the network might incorrectly associate or fail to associate training of other somewhat similar stopping situations.
Tesla could try to adjust the neural network hyper-parameters to better learn this stopping profile behavior, but that also can affect other behaviors in unexpected ways. The default behavior could be to "just" gather more training data of the desired motion profile in the various situations that it's inappropriate? But if there's always more training to be done, when should compute be allocated to architectural explorations?
Yeah, there's been improvements to perception neural networks, C++ control heuristics as well as the training and inference infrastructure/toolchain over the years, so most of those can be directly reused or extended for end-to-end. The control logic is the main thing discarded ranging from basic safe driving to following many traffic controls to handling cut-ins / merges to complex visibility creeping and utilizing median crossover regions. These probably already went through some amount of rewrites over the years to improve safety and to use of new types of predictions such as Occupancy as well as various attempts to make things smoother especially at highway speeds.
Even with all that existing effort, Tesla made the decision to restart at least the control portion from scratch with the hopes of iterating faster to the next level. But as you point out, Tesla isn't starting completely from scratch, so that's probably why they have been able to approach parity so quickly with all they've already learned and built.
Todd Burch
14-Year Member
FSD v12.1 Report from Customer (James Locke?) Ride
Seems like a report from James Locke. Excited to hear the turn signals are natural as it’s
a pain to do them myself to avoid looking silly!
From post on X:
“A highly respected (and critical) OG FSD beta tester went for a ride today in a Tesla employee's FSD Beta 12.1 car. Not naming names, but they are a true OG with 3+ years of experience and are critical of FSD's shortcomings. Here are their comments:
Just went for a drive with an employee with 12.1, very impressive.
Curb to curb zero disengagement drive on a beautiful sunny day on my test loop (they have only had a no-disengagement drive "a couple of times" on this loop in 3+ years).
V12.1 actually merged properly.
It was being conservative with speed when it didn't have any other traffic and it didn't see the first speed bump until the last second and it braked once it did which wasn't very smooth.
Turn signals are very natural.
Slowing to a stop was very human like.
Really a very impressive build. But again a beautiful day here.
The first speed bump I might have braked sooner but that was in the end it did brake in time.
It would pull up to and stop perfectly at the curb of my house. We engaged from the curb and it properly signaled and left the curb.”
Seems like a report from James Locke. Excited to hear the turn signals are natural as it’s
a pain to do them myself to avoid looking silly!
From post on X:
“A highly respected (and critical) OG FSD beta tester went for a ride today in a Tesla employee's FSD Beta 12.1 car. Not naming names, but they are a true OG with 3+ years of experience and are critical of FSD's shortcomings. Here are their comments:
Just went for a drive with an employee with 12.1, very impressive.
Curb to curb zero disengagement drive on a beautiful sunny day on my test loop (they have only had a no-disengagement drive "a couple of times" on this loop in 3+ years).
V12.1 actually merged properly.
It was being conservative with speed when it didn't have any other traffic and it didn't see the first speed bump until the last second and it braked once it did which wasn't very smooth.
Turn signals are very natural.
Slowing to a stop was very human like.
Really a very impressive build. But again a beautiful day here.
The first speed bump I might have braked sooner but that was in the end it did brake in time.
It would pull up to and stop perfectly at the curb of my house. We engaged from the curb and it properly signaled and left the curb.”
Last edited:
diplomat33
Average guy who loves autonomous vehicles
The last part about V12 pulling up to the curb is new and interesting. It suggests that Tesla is starting to train V12 for possible pick up and drop offs, a key feature needed for robotaxis.
powertoold
Active Member
Yea that was discussed in the Elon livestream: it was trained to pull over at the destinations but it wasn't consistent at the time
Artful Dodger
"Neko no me"
Omar striving for attention and early v12 access.
That link seems dead as well.
If this link still works, here's what James was thinking late yesterday afternoon. Maybe he got a v12 demo ride shortly after this post?
He may have pissed off someone again if he was told to delete the post of the v12 demo. Never trust a youtube click whore with a company secret.
Last edited:
If it is James Locke on his Test Loop 1, it's probably very exciting for him to no longer need to report this specific lack of turn signal for a merge after 3 years of pushing the video snapshot button and sending an email to the Autopilot team almost every test. Unclear if end-to-end was specifically trained for turn signals or maybe it was something it learned from the standard set of training data.
What controls does 12.x predict? Something related to accelerator/brake and steering wheel and turn signal. Maybe wipers too?
I wonder if we'll get updated feedback from the same set with an upcoming 12.2. Given the regressions relative to 11.x, it could be an indication of how quickly Tesla can fix issues in 12.1 released to Wave1 just a couple weeks ago. Presumably it hasn't expanded from employee testing due to some issues.
powertoold
Active Member
The V11 branch is already good enough to be useful for at least another 3-4 months. I hope Tesla takes the time to make a good first impression with V12. We need it to blow our socks off on release. That's how we can get viral FSD.
Frankly the only option available should be “Safe”. If you want to drive aggressively or be a block, you need to drive it yourself.
