Wiki Consumer AV - Status Tracking Thread

[diplomat33]

Livestream:

 

[Bitdepth]

Livestream:

Technical difficulties maybe?

 

[diplomat33]

Technical difficulties maybe?

It starts at 11am PST.

 

[diplomat33]

 

[diplomat33]

If anyone is interested, here is the link to the slides from Shashua's CES talk:

https://ir.mobileye.com/static-files/6b431df5-efd5-4653-95c7-b8cccc1c3b64

 

[scottf200]

If anyone is interested, here is the link to the slides from Shashua's CES talk:

https://ir.mobileye.com/static-files/6b431df5-efd5-4653-95c7-b8cccc1c3b64

1 hour youtube video here (posted 15 hours ago):
Mobileye: Now. Next. Beyond. CES 2024 Press Conference with Prof. Amnon Shashua

 

[diplomat33]

1 hour youtube video here (posted 15 hours ago):
Mobileye: Now. Next. Beyond. CES 2024 Press Conference with Prof. Amnon Shashua

Thanks. The one hour video is easier to watch than the livestream video I posted.

My thoughts on the CES talk:

I am very pleased with their roadmap and product portfolio. I think it bodes well for their approach that so many OEMs from different parts of the world are on board. According to the recent earnings call, the current OEMs represent about 34% of global auto production. And if the additional OEMs that are in talks with Mobileye also sign up, the total OEMs would represent 49% of global auto production. That's impressive! I can't really think of another AV developer that has the potential to scale the tech to so many car models and so many different geographies around the world as Mobileye. If Mobileye can deliver, it should really help scale autonomous driving tech to a lot of places and people.

I thought the parts about end-to-end were interesting. It seems Mobileye is taking a shot at Tesla and others who are trying the pure end-to-end approach (vision in, control out). I think Mobileye makes a valid point: getting one single monolithic DNN to 99.999999% reliability would be unprecedented. Of course, that does not mean that it can't happen but it would be very difficult. I do think doing end-to-end perception only and separate driving policy stack makes more sense since it would make things easier to build and validate and make your system less of a black box.

Overall, I really like Mobileye's approach of end-to-end perception with multiple redundancies, combined with crowdsourced maps and RSS policy. it sounds like a robust way to approach the problem of autonomous driving. And I like the incremental approach of building self-driving with vision-only, adding REM maps and RSS for added robustness and then adding radar-lidar for eyes-off. In this way, Mobileye hopes to deliver a range of systems from basic ADAS, to eyes-on navigate on pilot to full eyes-off everywhere without needing to radically change the hardware and software for each product. That makes a lot of sense.

I am intrigued by the DXP platform. It sounds like a great solution to the problem of delivering autonomous driving to so many different OEMs. With DXP, Mobileye can build a single autonomous driving system, and them let each OEM customize it to their specific needs. This saves Mobileye from the work of having to build different systems for each OEM or having the OEMs have to build their own autonomous systems from scratch.

The one thing that was probably missing was any sort of discussion of safety metrics. I would have liked to know what the current MTBF is to get a sense of how safe the system is right now. I also found it interesting that Shashua skipped his past talking point about "true redundancy" being semi-independent and achieving 10M hours MTBF. In fact, he did not even mention the 10M hour MTBF goal for eyes-off like he has done in the past. It makes me wonder if maybe that goal was unrealistic and so Mobileye has decided not to advertise their MTBF goal for eyes-off. And Shashua only showed very brief, edited clips of SuperVision in action that frankly did not really show much. The cynic might say that Mobileye is hiding safety data and only showing very brief edited clips because they are still very far from being able to do eyes-off. The presentation seemed to be more about making a case "on paper" for why their approach is right. The truth is that Mobileye likely still has a lot of work to do to get to eyes-off. But they have the advantage of being able to deliver eyes-on ADAS to a lot of customers while they work to get to eyes-off.

In conclusion, I am excited for Mobileye's work. I look forward to seeing SuperVision in the US, especially when they enable city streets, to compare it to FSD beta. The bottom line is that competition is good and the more reliable and safe eyes-on and eventually eyes-off systems are available, the better it will be for the consumer. We live in exciting times as autonomous driving in various forms will slowly but steadily scale to more and more places over the next few years.

 

[diplomat33]

Polestar 4 coming to the US with Mobileye SuperVision!

https://twitter.com/x/status/1745225565463015659

 

[scottf200]

I am intrigued by the DXP platform. It sounds like a great solution to the problem of delivering autonomous driving to so many different OEMs. With DXP, Mobileye can build a single autonomous driving system, and them let each OEM customize it to their specific needs. This saves Mobileye from the work of having to build different systems for each OEM or having the OEMs have to build their own autonomous systems from scratch.

I really loved this presentation and a feel for the challenges of the platform that is usable for multiple OEMs and then within one OEM for their take on the "feel".

 

[diplomat33]

I really loved this presentation and a feel for the challenges of the platform that is usable for multiple OEMs and then within one OEM for their take on the "feel".

Yes, Shai Shalev did a great job in the presentation. Mobileye seems to have a great solution to the challenge of delivering a safe, reliable autonomous driving product to multiple OEMs that each want something a bit different.

Side note: remember how Elon claimed that a major OEM was interested in licensing FSD? What's up with that? We have heard no updates on that in a while. It appears Elon was blowing smoke. It seems to me that Tesla is just trying to get FSD to work right, let alone trying to get FSD to meet the needs and requirements of other OEMs. Mobileye has over 10 OEMs that want their products!

That table relates to the discussion of how to do behavior prediction in your autonomous driving stack. Mobileye argues that their behavior prediction approach is the only one that meets all 4 requirements.

 

[Bladerskb]

Thanks. The one hour video is easier to watch than the livestream video I posted.

My thoughts on the CES talk:

I thought the parts about end-to-end were interesting. It seems Mobileye is taking a shot at Tesla and others who are trying the pure end-to-end approach (vision in, control out). I think Mobileye makes a valid point: getting one single monolithic DNN to 99.999999% reliability would be unprecedented. Of course, that does not mean that it can't happen but it would be very difficult. I do think doing end-to-end perception only and separate driving policy stack makes more sense since it would make things easier to build and validate and make your system less of a black box.

IMHO (which i'm more confident than ever on) I don't believe Mobileye will ever achieve true L4 on highway (full speed) or city in this decade. The recent presentation which honestly is just a regurgitation of 2020 and 2021 presentation sort of proves it. They are constrained by their support of their legacy system. It's similar to auto companies switching to EV from ICE, they have to support their ICE business. The same is the case here, Mobileye has to support traditional perception algorithms because there are OEMs who still want to use it to power basic ADAS features. In the same essence they are also constrained by their usual ASP. So they can't make a big powerful chip. This further restrains them on the kind of NN model they can run. So they end up using older CNN models that are far more efficient to run versus SOTA models that are FAR better, FAR more accurate but take alot more compute to run. Also they have limitation on how many cameras they can run these models on, resolution, area of pixels being processed and their fps. The use of intentions vs SOTA prediction network is also another choice made from their compute constraints.

Companies offering real L4 like Waymo have none of these limitations. They use BIG monolithic SOTA models on everything. Full resolution, all sensors, entire thing being processed, high fps, highly sophisticated models and finally they are not just running one Model. They are running several monolithic models to address a category.

So the entire mobileye argument is a complete redherring. Its borne from their constraints and L4 companies ain't running just one model.

The one thing that was probably missing was any sort of discussion of safety metrics. I would have liked to know what the current MTBF is to get a sense of how safe the system is right now. I also found it interesting that Shashua skipped his past talking point about "true redundancy" being semi-independent and achieving 10M hours MTBF. In fact, he did not even mention the 10M hour MTBF goal for eyes-off like he has done in the past. It makes me wonder if maybe that goal was unrealistic and so Mobileye has decided not to advertise their MTBF goal for eyes-off. And Shashua only showed very brief, edited clips of SuperVision in action that frankly did not really show much. The cynic might say that Mobileye is hiding safety data and only showing very brief edited clips because they are still very far from being able to do eyes-off. The presentation seemed to be more about making a case "on paper" for why their approach is right. The truth is that Mobileye likely still has a lot of work to do to get to eyes-off. But they have the advantage of being able to deliver eyes-on ADAS to a lot of customers while they work to get to eyes-off.

OfCourse it was PR. If you look back enough, Mobileye were saying they didn't need safety test drivers because they had "math/rss". I always thought that was complete none-sense. Its quite obvious they had the goal they had to hit Q1 2022 and they failed to do it and now its 2024 and they are still unable to produce a driverless system in their test city.

I'm of the belief that you need to produce a driverless system, even at small scale in other to replicate it at a larger scale (production cars, etc).

Today their MTBF on the highway in China has to be around 10 miles give or take. Based on the videos I have seen.
So they are very far from being anywhere close to offering a real L4 system on the highway, let alone on city streets.
Their EyeQ6 which I expected to have more advanced models, doesn't seem to. I just don't see how they are able achieve L4.

If Cruise was 3-4 years behind Waymo.
Mobileye is likely 6-7 years behind Waymo.
I don't see them achieve true L4 in any capacity this decade. Full stop.
The huge problem with them is, they are constrained by their small compute system. So they can't take advantage of the many breakthroughs in ML.
They are essentially stuck and have to make things work with their current setup.
Others like Waymo will essentially keep improving and ML will keep eating more and more of the stack.
Its not less ML that will bring us to L4 everywhere, Its MORE ML. As proven by the fact of Waymo going ML-first even in their planner.

 

[diplomat33]

Today their MTBF on the highway in China has to be around 10 miles give or take. Based on the videos I have seen.

You keep repeating this claim without any proof. How are you measuring MTBF? ME measures MTBF based collisions with injury. So you are saying that SV would crash every 10 miles? I am very skeptical. Show me these videos where SV would crash every 10 miles to back up your claim. I do know that ME completed that test with other Chinese cars where SV did 66 km with only one non safety intervention. SV did not crash every 10 miles in that video.

I would add that the goal is L4 that is also commercially viable at scale. Yes, Waymo has amazing big NN models that are solving L4. Waymo has the best L4 by far. But I believe you even slammed Waymo not too long ago for having a Google corporate mentality that often fails to commercialize tech advances. The point is that Waymo can have the best ML and the best L4 but if they fail to commercialize at scale, what was the point? I believe that consumer cars are the best way to commercialize AVs at scale. Robotaxis alone won't do it. I hope that once the L4 reaches a certain ODD, that Waymo will look to adapt their L4 to consumer cars. But the fact is that there are plenty of companies that either have L4 that is not commercially viable or have a commercially viable product but can't do L4. ME may fail to achieve L4 but I appreciate that they are at least trying an approach to reach L4 that is also commercially viable at scale.

 

[scottf200]

I don't believe Mobileye will ever achieve true L4 on highway (full speed) or city in this decade.

What do all these companies know that we don't know? How could they all be fooled? Certainly, from the outside it may look different than internal presentations, tech data, and actual implementation testing that these companies are doing (ie. "in production" in some columns below).

It sure seems like their environments can be more complicated that USA ones as well.
I think I saw a Polestar 4 is coming to the USA using Mobileye as well.

 

[diplomat33]

If you look back enough, Mobileye were saying they didn't need safety test drivers because they had "math/rss".

ME never said that. You are twisting their claims. In fact, ME uses safety drivers for all their testing. So clearly they believe in using safety drivers. What ME has claimed is that RSS guarantees that the driving policy will never cause an at-fault accident with injury (since RSS ensures the car always maintains a reasonable safety envelop around the car) and therefore any at-fault accidents with injury that do occur can only come from perception errors. So according to their claim, they would still need safety drivers but only for validating the MTBF of the perception stack, not for validating the driving policy. So no, they never claimed that never needed safety drivers.

 

[spacecoin]

Like @Bladerskb, I too remain very sceptical of MobilEye since the last two years. It seems premature to worry about "how to adopt the system to fit OEM branding". I remember a few years ago when EyeQ4 was supposed to be the answer to everything.

The released that Jerusalem drive in 2020, and it seems to me it's going backwards ever since. Is Navigation ZEEKR Pilot (NZP) live, and where can I find videos?

 

[diplomat33]

Is Navigation ZEEKR Pilot (NZP) live, and where can I find videos?

NZP has been live on Zeekr001 on highways for a year now. It is on over 100,000 cars in China now. And NZP will add city driving in Q1 of this year with full rollout expected towards the middle of this year.

Videos are available but they are on the Chinese internet. You have to go to weibo, I think, the Chinese google and search for them so they are hard to find.

That is why I am looking forward to SuperVision rolling out to cars in EU and US so that we can get easier access to videos when they are posted to youtube.

 

[diplomat33]

Like @Bladerskb, I too remain very sceptical of MobilEye since the last two years. It seems premature to worry about "how to adopt the system to fit OEM branding".

I totally understand the skepticism. We have not gotten any new full unedited videos. Shashua did not share any MTBF stats. And yes, we should be skeptical of pretty PR powerpoint slides. But the fact that so many OEMs are getting on board is a good sign. They would not sign up for SuperVision and Chauffeur unless they were convinced that it was the right product.

I am waiting to see how the rollout on city streets works. I hope we get some videos of that. I think seeing SuperVision in action on city streets in China could show us how it handles difficult cases. I am also looking forward to the rollout in EU and US to see how it works. And if we get new details say later this year from Polestar about how Chauffeur will work, that would also be a good sign. Let's see how things pan out before we declare ME a failure that will never solve L4.

I remember a few years ago when EyeQ4 was supposed to be the answer to everything.

Even when eyeQ4 was in production, I seem to remember ME touting eyeQ5 and eyeQ6 coming soon. So I am not sure eyeQ4 was ever promised as the "answer to everything". But ME has updated their compute requirements as new chips are developed. For example, I think at first they were testing Chauffeur with like 8 eyeQ4 because that is all they had at the time and now it is down to 3-4 eyeQ6H as they have more efficient chips.

 

[Bladerskb]

You keep repeating this claim without any proof. How are you measuring MTBF? ME measures MTBF based collisions with injury. So you are saying that SV would crash every 10 miles? I am very skeptical. Show me these videos where SV would crash every 10 miles to back up your claim.

极氪nzp-哔哩哔哩_Bilibili

bilibili是国内知名的视频弹幕网站，这里有及时的动漫新番，活跃的ACG氛围，有创意的Up主。大家可以在这里找到许多欢乐。

search.bilibili.com

All the videos are there going all the way back to July 2023 when highway NZP was released. Register an account so you can watch the videos in good quality (1080p). Also filter it by latest release.

I do know that ME completed that test with other Chinese cars where SV did 66 km with only one non safety intervention. SV did not crash every 10 miles in that video.

Averages work differently. Saying SV has an average MTBF of 10 miles doesn't mean that every SV will crash or require intervention every 10 miles. It's a statistical average, not a deterministic prediction for individual instances. So when I say the MTBF is "around 10 miles give or take," I'm saying on average, from all the videos i have seen. There has been an issue that required human intervention every 10 or so miles. This doesn't mean every vehicle will experience a problem exactly at the 10-mile mark. It's just the average experience across many vehicles over many miles. Some vehicles might travel 30 miles without incident, while others might have 2 issues after just 5 miles. When these distances are averaged, they might result in an MTBF of around 10 miles.

I would add that the goal is L4 that is also commercially viable at scale.

Yup and the only way to solve L4 and then scale it right now is using all the techniques that Waymo is using. I wish there were any other way, but there just isn't. There are no current shortcuts. No miracle drug. No breakthrough on the horizon. And even if there were a breakthrough that showed up, Mobileye wouldn't be able to use it cause their system is already constrained (Which is my point). They are locked in their current setup. Its similar to what we discuss about Tesla FSD hardware that uses 1.2 MP cameras with compromised views. Tesla is constrained on hardware, Mobileye is constrained on compute. The only working formula is hardware and compute that isn't constrained.

Yes, Waymo has amazing big NN models that are solving L4. Waymo has the best L4 by far. But I believe you even slammed Waymo not too long ago for having a Google corporate mentality that often fails to commercialize tech advances.

True and I still do. They're like Usain Bolt running High School track time.
But that doesn't mean the DNA makeup required to be Usain Bolt is different.
I believe big models are a requirement the same way I feel like front lidar and/or ultra resolution imaging radar are a current requirement.
Sophisticated simulation is also an requirement. When Mobileye unveiled their simulation, it was obvious this was something they just started working on.
And it wasn't surprising because they have been anti-simulation in the past (presentations)

The point is that Waymo can have the best ML and the best L4 but if they fail to commercialize at scale, what was the point? I believe that consumer cars are the best way to commercialize AVs at scale. Robotaxis alone won't do it. I hope that once the L4 reaches a certain ODD, that Waymo will look to adapt their L4 to consumer cars.

I'm there with you, I wish there were more companies challenging Waymo.

But the fact is that there are plenty of companies that either have L4 that is not commercially viable or have a commercially viable product but can't do L4. ME may fail to achieve L4 but I appreciate that they are at least trying an approach to reach L4 that is also commercially viable at scale.

I appreciate ME attempts, I'm not knocking them. I'm just analyzing them the same way I analyze any/every other company.
Same way I do with Tesla, Waymo, Cruise, etc.
Rather call a spade a spade than try to beat around the bushes. If you told me in 2020 that its 2024 and Mobileye didn't have a door-to-door city streets system in the hands of customers, i wouldn't believe you. But here we are.

 

[Bladerskb]

What do all these companies know that we don't know? How could they all be fooled? Certainly, from the outside it may look different than internal presentations, tech data, and actual implementation testing that these companies are doing (ie. "in production" in some columns below).

No one needs to be fooled. Remember these contracts has clauses on it.
For example if I remember correctly, Zeekr has NOT paid Mobileye at all or fully for SuperVision until they actually deliver the full functionalities.
So these agreements likely have the same clause that they won't have to pay until Mobileye delivers the full functionality that is agreed upon.
Payments is probably also broken down by functionality deliveries.

 

[diplomat33]

极氪nzp-哔哩哔哩_Bilibili

bilibili是国内知名的视频弹幕网站，这里有及时的动漫新番，活跃的ACG氛围，有创意的Up主。大家可以在这里找到许多欢乐。

search.bilibili.com

All the videos are there going all the way back to July 2023 when highway NZP was released. Register an account so you can watch the videos in good quality (1080p). Also filter it by latest release.

Thank you so much. This is great.

Averages work differently. Saying SV has an average MTBF of 10 miles doesn't mean that every SV will crash or require intervention every 10 miles. It's a statistical average, not a deterministic prediction for individual instances. So when I say the MTBF is "around 10 miles give or take," I'm saying on average, from all the videos i have seen. There has been an issue that required human intervention every 10 or so miles. This doesn't mean every vehicle will experience a problem exactly at the 10-mile mark. It's just the average experience across many vehicles over many miles. Some vehicles might travel 30 miles without incident, while others might have 2 issues after just 5 miles. When these distances are averaged, they might result in an MTBF of around 10 miles.

Yes, I know how averages work. This is why I wish Mobileye (and Tesla too) would release some official MTBF stats so we could get a more reliable measurement of their safety rather than looking at sample videos and trying to estimate an average.

The other question is how serious are the interventions? That matters. Are they small issues that are not safety critical or are they serious safety issues that would have caused a collision? If the MTBF of 10 miles you quote is for all "issues", even minor ones, that is very different from saying SV would likely cause a crash on average every 10 miles.

I just have a hard time believing that SV on highways is THAT bad, that it is actually requiring serious interventions on average every 10 miles. I mean, that MTBF is worse than FSD beta on highways. Are you saying FSD beta is better than SuperVision on highways?

Yup and the only way to solve L4 and then scale it right now is using all the techniques that Waymo is using. I wish there were any other way, but there just isn't. There are no current shortcuts. No miracle drug. No breakthrough on the horizon.

I agree. I have no doubt Waymo will solve L4. The question is can they be commercially viable and how fast can they scale.

The only working formula is hardware and compute that isn't constrained.

Sure but in the real world, there are constraints like cost. You could put $100k worth of sensors and compute on a car and solve L4 but that won't work for commercially viable product. So you still need to reduce cost, reduce sensors etc... to make the system commercially viable. So there are constraints whether we like it or not. Ultimately, to truly solve autonomous driving, we will need a solution that both delivers safe and reliable L4 and also does it at an affordable cost and efficient compute.

If you told me in 2020 that its 2024 and Mobileye didn't have a door-to-door city streets system in the hands of customers, i wouldn't believe you. But here we are.

Yeah, I thought ME would have deployed door to door city streets awhile ago too.