Navigate on Autopilot is Useless (2018.42.3)

[DrDabbles]

...and had pretty much the same results as yesterday.

Very interesting. I assume you've done the bug report thing. It's pretty crazy how vastly different versions behave, and how serious regressions creep in. But, this is pretty much what you'd expect of a ML system approaching its maximums.

Also, the car... both cars, actually, behave perfectly fine with NavOnAP disabled and normal AP working. No phantom braking, and obviously no changing lanes into other vehicles.

I think it's pretty clear at this point that EAP is a simple line follow robot with some radar input. NoAP appears to be the actual semi-autonomous system, and these "corner" cases aren't nearly rare enough and the system doesn't handle then nearly smart enough. I'm interested to see what the event on the 22nd is all about, and whether they give demonstration rides of a fully autonomous system or not.

I don't feel like a calibration issue could be the cause...

The fact that two different vehicles did the same thing pretty much guarantees it's not a calibration issue.

Definitely seems like a YMMV situation, since other's are reporting glorious results... so, who knows. I'm definitely not losing my mind, since I had passengers today and yesterday who witnessed the same shenanigans.

I've had good results, but nothing I'd ever consider glorious. Then again, I disable NoAP/EAP when more than one lane is merging, when vehicles around me are driving extra crazy, and in situations I think are probably too complex. It's not a 100% solution. It's probably a 75-80% solution in simple cases and heavily traveled areas. But it's got a long way to go before the thing can actually drive itself.

 

[scottf200]

Dr. Camilo Ortiz @DrCamiloOrtiz· 16h
Im amazed. My car drove me practically all the way home, with no input from me, changing lanes, merging on and off of 3 highways, in rush hour, in the rain, in NYC traffic!!! Stunned. @tesla @elonmusk

 

[timewasted]

I don't have 2019.8.5 so I can't try out the new amazing "@wk057 is a threat and must be dealt with" neural net, but 2019.8.4 is still as worthless as the last time I posted here about NoA. If I'm sitting in the middle lane on the highway with my cruise set to 75mph with the person in front of me going 65mph and the lane to my left is completely empty, my car will happily continue following this person regardless of what my aggressiveness setting is at. It constantly suggests I merge to the right to prepare for my exit despite said exit still being 4+ miles away. Merging on and off freeways is still scary as hell because the car doesn't seem to care much about cars beside it as the lanes start to condense.

I'm glad other people are finding it useful, but it's just simply been a liability for me. I'll give it a try again when I finally get 2019.8.5, but probably only when I'm driving alone in the car.

 

[VT_EE]

^



I thought this as well, since it definitely felt odd that there would be such a regression. For unrelated purposes, I drove a completely different car (friend's late-2018 S that just got the update) this morning on part of the same route, and had pretty much the same results as yesterday.

Also, the car... both cars, actually, behave perfectly fine with NavOnAP disabled and normal AP working. No phantom braking, and obviously no changing lanes into other vehicles.

I don't feel like a calibration issue could be the cause of it attempting lane changes into vehicles it's clearly displaying on the visualization, though, since it can obviously see them if it's showing them on the screen. Like, it would draw the lane change path through the other vehicle on the screen. Going to have to setup a cabin GoPro next time I do a longer trip in the 3 to see the screen and surroundings in the frame.

Definitely seems like a YMMV situation, since other's are reporting glorious results... so, who knows. I'm definitely not losing my mind, since I had passengers today and yesterday who witnessed the same shenanigans.

I used NoA today for about 2 hours of highway driving today with moderate traffic at an average of 64 mph. I paid close attention whenever it was doing lane changes to make sure it wouldn’t move into vehicles. It worked perfectly every time with my only complaint being it needed to turn sooner after initiating the blinker. There were quite a few times the system waited for people to get out of the blind spot before moving lanes, even after it had turned the blinker on. It was totally opposite of your experience. I don’t doubt your story and am very curious what would cause these highly divergent test results.

 

[mhan00]

Don’t think this particular visualization matters too much - mine happily draws paths through cars, but it also respects their boundaries a lot more than yours seems to ...

Same for me. My car will show the lane change line crossing over into in adjacent vehicle, but the car won’t actually move to complete the lane change until the space is clear. Before this new update, it would almost always slow to get behind the car in the blind spot, even if there was plenty of space in front to accelerate to create space which is how I’d usually do it. I usually ended up hitting the accelerator myself to clear the car from my blind spot, and when there was space the car would change lanes. When I used it a couple nights ago, it actually accelerated itself a couple times, which was nice.

 

[mhan00]

I don't have 2019.8.5 so I can't try out the new amazing "@wk057 is a threat and must be dealt with" neural net, but 2019.8.4 is still as worthless as the last time I posted here about NoA. If I'm sitting in the middle lane on the highway with my cruise set to 75mph with the person in front of me going 65mph and the lane to my left is completely empty, my car will happily continue following this person regardless of what my aggressiveness setting is at. It constantly suggests I merge to the right to prepare for my exit despite said exit still being 4+ miles away. Merging on and off freeways is still scary as hell because the car doesn't seem to care much about cars beside it as the lanes start to condense.

I'm glad other people are finding it useful, but it's just simply been a liability for me. I'll give it a try again when I finally get 2019.8.5, but probably only when I'm driving alone in the car.

Lmao. Iirc, wk057 was offered a job by Tesla and he turned them down. Maybe Elon took it personally.

 

[scottf200]

Check out a few minutes of this great demo with some dashcam coverage thrown in.

It goes on-ramp to off-ramp and does a few lane changes. Impressive.

Here are his options:

Start at 2:23

 

[Electroman]

It makes very nervous on that video the car driving on AP at 80 mph, on a not so straight and not so well marked lanes. I would never do that.

In fact with NoA I would restrict it to 5 mph over the speed limit.

 

[DrDabbles]

Just got back from driving around even more, and I can confirm what @wk057 said. EAP behaves much more sensibly than NoAP. So, this would mean that the neural net responsible for viewing and deciphering the environment is working fine, but the control system responsible for driving the car and following GPS is pretty much doing its own thing.

Yanking the car around to center itself in what it thinks are lanes is ridiculous again, lane change decisions are apparently being made without regard for whatever else the car is doing. So, I was in a sharp right curve on the highway, the car randomly slowed and sped up, then slowed way down and wanted to change to the right lane, which further sharpened the curve and caused the car to complain about tight turns and slow down even more.

Phantom braking events while driving past offramps is unacceptable. And in a situation where a highway split straight and right, and there was an exit I needed to take that was further right, the car jerked itself left and right on the road repeatedly while traffic behind me tried to dodge what the car was doing.

So, I'd say that as of now, to consider this "epic" as Elon called it online is a total mischaracterization. It's interesting, it's a preview of where Tesla wants the technology to go, but it's not at all ready to be considered "driving me to work with no input".

 

[ivengo]

My experience with 2019.8.5 is very smooth. I have not detected any problems in several hours I've driven it. It did NOT perform dangerously at any time.

 

[scottf200]

Just got back from driving around even more, ...

Are you on 2019.8.5?

 

[DrDabbles]

Are you on 2019.8.5?

Yes

 

[GeoX750]

My experience with 2019.8.5 is very smooth. I have not detected any problems in several hours I've driven it. It did NOT perform dangerously at any time.

Seriously, no hyperbole ?

I just got back from a drive and it was scary every few minutes, especially with cars merging into the lane, merging into traffic/lane changes in traffic, approaching on/off ramps and twitchy steering that I have not experienced before.

Not ready for Chicago traffic conditions.

 

[scottf200]

Another nice example of it working well.

 

[cybergates]

the lane changes are done in a safe manner on 2019.8.5 I found (model 3) driving around so cal. The only issue is unnecessary lane changes esp wanting to get into the carpool lane even with speed based lane change disabled and esp problematic when its wanting to move left and the exit is less than < 1-2 mile away. Start by keeping lane based speed changes to mild or disabled.

 

[S4WRXTTCS]

I find it kinda of fascinating that it seems like those with good experiences are posting videos, and people with bad experiences are generally not.

When I test 2019.8.5 I should setup a camera so I can record a video to get more representation for us haters.

My video is probably going to look like this

Car: Move to the right lane
Me: But, I don't want to move to the right lane. This middle lane is just fine, and don't bug me again about it
Car: For navigation purposes please move over
Me: No
Car: I said move over
Me: NO!!!
Car: I don't care how many times you cancel it. I'm simply going to keep bugging you.
Me: Okay, fine, But, promise not to do that hunting thing you do at every onramp point. You make us look like complete idiots.
Car: Moves over to the right lane
Road: There goes the line as it's a merge area.
Car: Moves to the right
Me: ARRGGH, you promised not to do that!!
Car: HAHA. You think I follow ultra detailed maps??? This isn't California!!
Me: Screw you NoA, and just to punish you I'm going to switch to TACC
TACC: I win again

 

[electronblue]

I find it kinda of fascinating that it seems like those with good experiences are posting videos, and people with bad experiences are generally not.

When I test 2019.8.5 I should setup a camera so I can record a video to get more representation for us haters.

My video is probably going to look like this

Car: Move to the right lane
Me: But, I don't want to move to the right lane. This middle lane is just fine, and don't bug me again about it
Car: For navigation purposes please move over
Me: No
Car: I said move over
Me: NO!!!
Car: I don't care how many times you cancel it. I'm simply going to keep bugging you.
Me: Okay, fine, But, promise not to do that hunting thing you do at every onramp point. You make us look like complete idiots.
Car: Moves over to the right lane
Road: There goes the line as it's a merge area.
Car: Moves to the right
Me: ARRGGH, you promised not to do that!!
Car: HAHA. You think I follow ultra detailed maps??? This isn't California!!
Me: Screw you NoA, and just to punish you I'm going to switch to TACC
TACC: I win again

Any thoughts on the (potentially) AP2.5’s apparent folllowing of a diamond marker on the road as the lane line and ending up on the curb?

Because it sure looks like to me the car got confused by the diamond marking being the right lane marker and trying to cross between the left lane marker and diamond... thus ending up losing sight of the real right lane marker and then repositioning itself between the curb as the left lane marker and the actual left lane marker as the right lane marker... and then simply running out of space.

Can be a coincidence of course but given the sticker hack looks like a very potential reason too.

 

[S4WRXTTCS]

Any thoughts on the (potentially) AP2.5’s apparent folllowing of a diamond marker on the road as the lane line and ending up on the curb?

My understanding was the sticker hack was reported to Tesla, and they fixed it that particular one. The video was taken late last year so the car probably didn't have the update. So it's possible.

Not that it required a sticker hack to do this. People have crashed into close barriers under AP with (regardless of AP1 or AP2). So sometimes the system just loses the lines, and the driver fails in their job to oversee it. When I'm close to barriers I always turn off AP since I don't want to chance it.

On this particular case I don't think we can assume the car was on AP. It's usually a given with an expensive Model S, but lots of people didn't get it for the 3. Hence why they came out with the lower priced AP package (or at least Elon claims this). To me it looks like the driver simply wasn't paying attention (like texting), and unintentionally drifted to the left. It doesn't appear to do a sudden jerky movement like I've experienced when it gets confused by lines. It also has a nice clear unobstructed yellow line, and diamonds are pretty common.

 

[electronblue]

People have crashed into close barriers under AP with (regardless of AP1 or AP2). So sometimes the system just loses the lines, and the driver fails in their job to oversee it. When I'm close to barriers I always turn off AP since I don't want to chance it.

Is there any instance of this happening on AP1 where the lane lines did not end up in the barrier? I do recall the video where the lane lines lead directly into concrete but that was a very different scenario to being distracted by a secondary road marking.

 

[mhan00]

Pretty impressive moment for me tonight where NOA was performing a lane change and when the car had just breached the lane I saw a motorcycle rapidly approaching from behind in the target lane, probably going 20-30 mph faster than me. I was about to cancel the lane change via the steering wheel to let the crazy bastard pass when AP did it for me, manuevering back into the original lane, letting the motorcycle zip by, then re-initiating and completing the lane change.