Autonomous Car Progress

[bd7349]

Speaking of the vector space bird's eye view, here is a good article that describes how it might work:

Tesla Bird's Eye View Explained, Tesla To Offer Bird's Eye View In FSD Package - VehicleSuggest

Have you seen this? 3D Packing for Self-Supervised Monocular Depth Estimation [CVPR 2020]

Likely that Tesla is going for a high fidelity 3D vector-space representation of the world similar to that. It seems like the only solution possible for a birds eye view that’s useful for parking since it would need to accurately show what the car sees or the feature would be useless. The current vector-space representation shown on the screen is pretty basic and shows none of that at the moment.

It’s worth noting that the high fidelity 3D representation was done with the training of video data (explained here in their CVPR video, different from the one above) which we know is what Tesla aims to do/is doing. Also noteworthy is that they did this with the data from only one camera. Tesla has 8 cameras to utilize depth data from, so it wouldn’t surprise me if their 3D model would be even more detailed and accurate than what is seen in the 3D packing video (which again, it would have to be if it’s to be useful for parking and other tight maneuvers).

The Autopilot visualizations today already are vector space rendering of selected neural network outputs, and this allows the driver to move the virtual camera around with the touch screen. I believe green found Godot Engine for MCU2 that could be used to render these visualizations.

The birds-eye viewpoint already happens when the vehicle is placed in park (i.e., virtual camera is moved to above the vehicle instead of behind), so I would guess the rewrite having the neural network process all cameras allows for more consistent outputs of parking lines, curbs, parking barriers, adjacent vehicles, etc.

There's a few advantages of vector space as opposed to traditional real-time camera feeds in a birds eye view perspective:
- avoids camera inconsistency, e.g., color, calibration, distortions
- dynamic camera positions, e.g., zooming to wheels close to curbs
- permanence of static objects, e.g., estimating obscured barrier distance

You’re correct, although the current birds eye view vector-space representation is pretty basic and non-detailed. What I’ve linked above is exactly what you’re describing in a highly detailed 3D vector-space representation of the world (no camera inconsistencies, dynamic camera positioning, object permanence).

 

[banned-66611]

Waymo's robotaxi service is now available to the general public for fully driverless rides.

Waymo finally launches an actual public, driverless taxi service

 

[Buckminster]

Waymo's robotaxi service is now available to the general public for fully driverless rides.

Waymo finally launches an actual public, driverless taxi service

Exactly. We barked up that tree for way too long (sigh). Gives a false sense of victory being close — a tantalizing local maximum — but reality is just too messy & weird. Our new system is capable of driving in locations we never seen even once.

 

[banned-66611]

Exactly. We barked up that tree for way too long (sigh). Gives a false sense of victory being close — a tantalizing local maximum — but reality is just too messy & weird. Our new system is capable of driving in locations we never seen even once.

Who is "we"? Tesla?

Musk said they would start a limited beta in certain areas only.

The Waymo system could probably handle most of the US and Europe already, they are just being extremely careful with deployment. Don't want to end up like Tesla with a string of fatal accidents.

 

[diplomat33]

The Waymo system could probably handle most of the US and Europe already, they are just being extremely careful with deployment. Don't want to end up like Tesla with a string of fatal accidents.

I am impressed that Waymo is going full driverless for their public rides. IMO, removing the safety driver is a good indicator that you have "real FSD" because it means that your FSD is ready to drive on its own. Like a flight instructor letting a new pilot fly solo. And letting the general public ride in fully driverless cars indicates that Waymo has high confidence in their system's safety. IMO, it is also a big indicator that the Waymo approach does really work.

I reserve judgment for Europe since driving there is very different than the US and I don't think Waymo has done much testing in Europe. But I do think Waymo could handle most driving in the US now if they wanted to. I say "most" because I am sure there are cases that Waymo cannot do yet. I realize Waymo does not have perfect FSD. Waymo is just being cautious to avoid unnecessary accidents. They will collect more data and make sure that their system is fully safe before deploying to new areas.

The reason I think the Waymo Driver could handle most US driving is because a lot of the driving that the Waymo Driver can handle now will be the same in lots of other places. There will be overlap. For example, if you can handle say a pedestrian crossing in front of your car in one city, you can probably handle it in other cities too. If you can handle cut ins in one city, you can probably handle cut ins in other places. If you can handle a four way intersection in one city, you can probably handle it other cities. And Waymo is testing in 25 US cities so they have a good sample size of driving in the US. With enough data, there system will be handle most driving anywhere. Perception is already done. The real challenge is planning and driving policy. Once you have planning and driving policy that is smart and flexible enough, it will work in lots of different places. For example, Waymo might encounter a new situation that it did not encounter in Phoenix, but if it has smart driving policy that knows to drive conservatively, yield to other vehicles etc... it will probably be fine.

Again, I am not suggesting that Waymo's FSD has completely arrived. But yes, I think it could probably handle most driving in the US now. Waymo just needs to solve those remaining edge cases and perfect the planning and driving policy more and they will eventually have L5 IMO.

 

[diplomat33]

I think I found a new Waymo video from a recent drive. It was posted just a few hours ago on reddit. It is a short drive but it is completely driverless and at night.

4 Minute Full Video in Waymo One - No Driver; Short Trip : waymo

 

[Knightshade]

The Waymo system could probably handle most of the US and Europe already

Given they still haven't gotten it working perfectly in places with actual weather other than "clear and sunny".... no.

(apart from not having the super high end HD maps everywhere they rely on)

, they are just being extremely careful with deployment. Don't want to end up like Tesla with a string of fatal accidents.

And by "string" you mean "one, ever, using it in a place it was intended to be used....by a dude busy on his phone who still only died because the crash barrier hadn't ever been repaired....and with billions more driven miles than Waymo"

 

[daniel]

The reason I think the Waymo Driver could handle most US driving is because a lot of the driving that the Waymo Driver can handle now will be the same in lots of other places. There will be overlap. For example, if you can handle say a pedestrian crossing in front of your car in one city, you can probably handle it in other cities too. If you can handle cut ins in one city, you can probably handle cut ins in other places. If you can handle a four way intersection in one city, you can probably handle it other cities.

What Waymo has accomplished and what it is doing now is impressive, and I am encouraged by it. But it's worth noting that while pedestrian crossings, cut-ins, and four-way intersections may be essentially the same in most or all US cities, they are operating in a small suburban area of one of the driest places in the country. It never snows, it seldom rains, and it's neither central-city nor rural, both of which present their own challenges.

Also, while the cars are driverless, there are remote operators who can send high-level commands when the car encounters a problem. So it's more like Level 4 than Level 5. A privately-owned car with this technology could transport you while you nap, and wake you up when needed, but could not go off entirely empty to fetch the kids from school.

That said, it would probably meet my needs just fine here.

 

[diplomat33]

What Waymo has accomplished and what it is doing now is impressive, and I am encouraged by it. But it's worth noting that while pedestrian crossings, cut-ins, and four-way intersections may be essentially the same in most or all US cities, they are operating in a small suburban area of one of the driest places in the country. It never snows, it seldom rains, and it's neither central-city nor rural, both of which present their own challenges.

To be clear, that small suburban area that is so dry and sunny is just where Waymo has launched a ride-hailing service for the public. That is just where they have commercialized their FSD. But Waymo is actively testing their FSD in lots of other places, including some very rainy and snowy places. Waymo has autonomous cars driving around with safety drivers in over 25 cities in the US.

Also, while the cars are driverless, there are remote operators who can send high-level commands when the car encounters a problem. So it's more like Level 4 than Level 5. A privately-owned car with this technology could transport you while you nap, and wake you up when needed, but could not go off entirely empty to fetch the kids from school.

Yes, Waymo is level 4. I don't think anyone is claiming that it is L5. But that has to do with the geofenced service area not any remote operator. The cars are fully self-driving with no human needed inside the service area.

And Waymo cars do drive around entirely empty to fetch new passengers. So they already do what you say. If the school were inside the service area, the cars could absolutely go off entirely empty to fetch your kids from school and bring them home. That is what you seeing in that video actually. The car fetched a passenger empty, drove them somewhere and dropped them off with no driver inside.

 

[daniel]

To be clear, that small suburban area that is so dry and sunny is just where Waymo has launched a ride-hailing service for the public. That is just where they have commercialized their FSD. But Waymo is actively testing their FSD in lots of other places, including some very rainy and snowy places. Waymo has autonomous cars driving around with safety drivers in over 25 cities in the US.

Yes. And that is great, and they are making progress, and I am encouraged by this. But the claim "They are now driverless for public ride-hailing" is tempered by the fact that it's only in a suburban area with perfect weather.

Yes, Waymo is level 4. I don't think anyone is claiming that it is L5. But that has to do with the geofenced service area not any remote operator. The cars are fully self-driving with no human needed inside the service area.

And Waymo cars do drive around entirely empty to fetch new passengers. So they already do what you say. If the school were inside the service area, the cars could absolutely go off entirely empty to fetch your kids from school and bring them home. That is what you seeing in that video actually. The car fetched a passenger empty, drove them somewhere and dropped them off with no driver inside.

The Waymo robotaxis can drive with nobody in the car precisely because they have the remote operator available. What I said was that if you could buy that car, you could not send the car off with nobody in it. The car is capable of pulling over safely and calling for help (or potentially waking up a sleeping driver) but it needs one or the other: A person available in the car or by remote.

So at this point if you owned one of these cars off of the Waymo remote monitoring network, you could sleep in it if you were in that small part of Phoenix, but you could not send it off empty. And outside of that small area of Phoenix, you would still need an alert safety driver. So in its geofenced area it's Level 4, and outside that area it's Level 3.

(I'd still buy one! I'd jump at the opportunity to buy a Level 3 car right now.)

 

[DanCar]

Wonder how long before they break out of the 5x10 mile box? I would caution those who say it is now open to the public. A work acquaintance in the area says he can not ride and others on reddit have also stated they can't ride the service yet. But there a couple who say they have ridden. It is still a very very small scale operation from my perspective, but still impressive they are finally actually doing driverless, although I would discount this if we do NOT get a good stream of videos. Too many times waymo said one thing, when the truth was something else.
Waypoint - The official Waymo blog: Waymo is opening its fully driverless service to the general public in Phoenix

In the near term, 100% of our rides will be fully driverless.

To me this suggest the driverless rides will remain very small scale for a while. Perhaps limited by the number of people remote monitoring and or remote operator infrastructure.

 

[Knightshade]

To be clear, that small suburban area that is so dry and sunny is just where Waymo has launched a ride-hailing service for the public.

Because it's the only area their service is good enough to do so.

They didn't pick that place by accident- they picked it because it was easier than most other places

The fact it's only offered there tells you their solution is not good enough for everywhere yet.

T
That is just where they have commercialized their FSD. But Waymo is actively testing their FSD in lots of other places, including some very rainy and snowy places.

I feel like we've already been over this previously- where I cited Waymo is doing that testing elsewhere because currently the system does not work as well in those places.

So they still have a ways to go in places without perfect weather and perfect HD maps of the local situation.

 

[diplomat33]

Because it's the only area their service is good enough to do so.

They didn't pick that place by accident- they picked it because it was easier than most other places

The fact it's only offered there tells you their solution is not good enough for everywhere yet.

I just listened to the TC interview with the COO and she mentions government regulations as the number reason why they chose AZ to launch Waymo One. Specifically, the governor of AZ signed an Executive Order allowing both the testing and deployment of autonomous ride-hailing services in the State.

Is it possible that the nice weather was also a factor? Sure. But nobody is suggesting that Waymo has perfect L5. That is why Waymo is testing in other areas where there is FSD is not as good yet. I am just saying that Waymo has driverless cars in Phoenix which is more than any other company can claim. I don't see Tesla deploying any driverless cars at all.

Mark my words: we will see Waymo launch ride-hailing in other cities (pending regulations).

 

[S4WRXTTCS]

Once again it looks like people are so focused on the fairy-tale of L5 driving that they can't see what an accomplishment this is.

This is true L4 driving in a geo-fenced area, and its impressive.

At first glance it looks to be an easy environment, but what do human do in easy environments? They do stupid things that complicates the easy environment. I would fully expect pedestrians to jay walk across the road at all hours for example.

45 hour speed limit areas in uncontrolled environments can be pretty dangerous. You'll have one car traveling 55mph while someone else is doing 35mph. You'll have people pulling u-turns at random across all the lanes.

It's also in Arizona and if you've ever driven in Arizona you know its only a matter of a small amount of time till some other driver tries to kill you.

Sure as someone from Seattle I'm not too impressed with the whole lack of rain/fog/etc, but eventually Waymo will start to expanding it to places where it does rain. I don't expect that it will be allowed in any really challenging environment for quite a while simply due to liability, and more importantly public perception.

Any injury/fatality accident will cause all kinds of media coverage, and scrutiny. The last thing Waymo wants is to deal with that so they're going to be extremely cautious. Some smaller player will have to do the more daring stuff.

Even in a limited use fashion its still going to be pretty popular because of all the benefits an autonomous taxi service provides.

 

[Tam]

..."They are now driverless for public ride-hailing" is tempered by the fact that it's only in a suburban area with perfect weather...

By the way, when there've been fatal Autopilot accidents, Elon Musk still refused LIDAR reasoning that it does not function well in bad weather.

Notice that every single of the 3 Autopilot accidents has happened in bright daylight with very good weather.

Any incremental gain is good even if it can't do in bad weather. One thing at a time until perfections.

 

[NHK X]

I just listened to the TC interview with the COO and she mentions government regulations as the number reason why they chose AZ to launch Waymo One. Specifically, the governor of AZ signed an Executive Order allowing both the testing and deployment of autonomous ride-hailing services in the State.

Is it possible that the nice weather was also a factor? Sure. But nobody is suggesting that Waymo has perfect L5. That is why Waymo is testing in other areas where there is FSD is not as good yet. I am just saying that Waymo has driverless cars in Phoenix which is more than any other company can claim. I don't see Tesla deploying any driverless cars at all.

Mark my words: we will see Waymo launch ride-hailing in other cities (pending regulations).

What’s the timeframe to mark your words with?

 

[diplomat33]

What’s the timeframe to mark your words with?

I am being conservative but I will say 5 years. I say in 5 years, Waymo will have a Waymo One driverless service in at least one other city in the US, in addition to Chandler, AZ. And I think the next Waymo One service will probably be in CA.

 

[Buckminster]

I am being conservative but I will say 5 years. I say in 5 years, Waymo will have a Waymo One driverless service in at least one other city in the US, in addition to Chandler, AZ. And I think the next Waymo One service will probably be in CA.

Are you kidding me? And these are world leaders?... Do think think that Waymo One will generate more than $1k revenue per annum in the next 5 years?

Apparently I am quite bullish on Waymo...

 

[diplomat33]

Are you kidding me? And these are world leaders?... Do think think that Waymo One will generate more than $1k revenue per annum in the next 5 years?

Apparently I am quite bullish on Waymo...

It's just a guess. I only said 5 years so that I am virtually guaranteed to be right.

I believe Waymo's FSD is technically good enough that they could expand to every city in the US now, with safety drivers. But I have no idea what Waymo's specific expansion plan is or what regulations will do.

Yes, I think Waymo One will generate a lot more than a $1k/year in revenue in the next 5 years. Waymo is the world leader in FSD because their FSD is the best by far. But expansion takes time. As the Waymo COO and CEO have explained, it is not just improving the FSD tech, it is also working with local regulations as well as getting feedback from all stakeholders to ensure a good customer experience.

 

[SalisburySam]

Waymo just needs to solve those remaining edge cases and perfect the planning and driving policy more and they will eventually have L5 IMO.

...but isn’t that the hard part?