Tesla replacing ultrasonic sensors with Tesla Vision

[enemji]

I have a 2020 MS LR+. For the past two years, every time I install an update I check to see if they fixed it - but they did not.
For some reason they randomly decide to go low speed swiping on a bright sunny day, without reason. Could not find a particular pattern - clean, dirty windshield, location, etc. It swipes a few times (with the horrible screech on a dry windshield) and it stops.
On the other side, the moment there is a single drop of rain the wipers go max speed.
I gave up on them and control them manually using the stalk. Now, with the radar gone, their behavior makes the AP unusable.

Did you ever get your windshield replaced or tinted or some other treatment?

 

[stopcrazypp]

I have a 2020 MS LR+. For the past two years, every time I install an update I check to see if they fixed it - but they did not.
For some reason they randomly decide to go low speed swiping on a bright sunny day, without reason. Could not find a particular pattern - clean, dirty windshield, location, etc. It swipes a few times (with the horrible screech on a dry windshield) and it stops.
On the other side, the moment there is a single drop of rain the wipers go max speed.
I gave up on them and control them manually using the stalk. Now, with the radar gone, their behavior makes the AP unusable.

Maybe it is worse on the Model S, but I haven't had dry wipes on my Model 3. Note the priority for the wipers are to clear the camera, sometimes if spray gets up there, it will wipe even if your view is clear. And the flip side is true also (if spray doesn't get on camera sometimes it doesn't wipe even if your view is not clear).

 

[stopcrazypp]

I wasn't going to reply to this but then I hit that last sentence.
What connection are you making between the presence or absence of radar, and the performance of Tesla auto wipers on Autopilot?

Probably talking about the fact under Vision, it forces auto wiper and also auto high beams.

 

[JHCCAZ]

Probably talking about the fact under Vision, it forces auto wiper and also auto high beams.

Okay maybe so, but I don't see that as being directly connected to radar (at least not that radar), which didn't help with the detection of pedestrians, driving paths or (stationary) unlit objects at night.

It's true that the removal of radar happened around the time that a great deal more functionality was being assigned to camera vision - but that was going to happen anyway, and would have mandated the AP control of wipers and brights just the same.

Anyway, I know it wasn't your post, and thanks for that answer which makes some sense. I just wasn't getting the connection.

 

[Dewg]

I'm starting to wonder about the interdependence of all the systems in Tesla. I think we're finding people with FSD issues that are finding they have GPS issues. I wonder if wipers and high beams are symptoms too. People that have wiper problems, with poor speed choices, and especially where it cleans an already clean and dry window, do these people also have FSD problems?

 

[Boza]

Okay maybe so, but I don't see that as being directly connected to radar (at least not that radar), which didn't help with the detection of pedestrians, driving paths or (stationary) unlit objects at night.

It's true that the removal of radar happened around the time that a great deal more functionality was being assigned to camera vision - but that was going to happen anyway, and would have mandated the AP control of wipers and brights just the same.

Anyway, I know it wasn't your post, and thanks for that answer which makes some sense. I just wasn't getting the connection.

Nothing special on the windshield - it is the way it came from the factory.
When I had the radar the wipers and high beams worked independently from the AP. So, I did not use the auto options.
With Tesla Vision AP forces auto wipers and auto high beams. So, by removing the radar I have to endure dry wipes or crazy max speed wipers plus high beams that flash the road signs in order to use the AP. So, I am not using the AP anymore.
The price of the car creates certain expectations. Not being part of experiments and having working basic features are part of those expectations.

 

[AlanSubie4Life]

Nothing special on the windshield - it is the way it came from the factory.
When I had the radar the wipers and high beams worked independently from the AP. So, I did not use the auto options.
With Tesla Vision AP forces auto wipers and auto high beams. So, by removing the radar I have to endure dry wipes or crazy max speed wipers plus high beams that flash the road signs in order to use the AP. So, I am not using the AP anymore.
The price of the car creates certain expectations. Not being part of experiments and having working basic features are part of those expectations.

Post some videos of this. Definitely very strange though plenty of similar reports. I’ve not had this problem at all, Model 3.

 

[Boza]

Post some videos of this. Definitely very strange though plenty of similar reports. I’ve not had this problem at all, Model 3.

Will do. But you can also do a YT search - quite a few of them…

 

[AlanSubie4Life]

Will do. But you can also do a YT search - quite a few of them…

Yeah I know but often no video of outside of windshield, etc. or at night, etc. plus this is your report so good to see the actual vehicle doing it. The stuff I have seen on YouTube is not normal behavior.

 

[JHCCAZ]

Nothing special on the windshield - it is the way it came from the factory.
When I had the radar the wipers and high beams worked independently from the AP. So, I did not use the auto options.
With Tesla Vision AP forces auto wipers and auto high beams. So, by removing the radar I have to endure dry wipes or crazy max speed wipers plus high beams that flash the road signs in order to use the AP. So, I am not using the AP anymore.
The price of the car creates certain expectations. Not being part of experiments and having working basic features are part of those expectations.

I can understand the frustration of having certain aspects of the car's systems having changed/ ldegraded behavior. Without denying that this is true. nor trying to excuse it, , I think it's perceived as even worse when you've gone through the first learning learning curve and have already adapted to the idiosyncrasies of your new car. Then to have it changed is worse than the experience of a brand-new owner who is learning on that already-changed behavior.

‐------‐----------------------
More thoughts on auto-wiper functionality, and my conclusion that Tesla has probably made the correct choices. For those who are interested, I offer the following line of reasoning:

My experience with auto wipers has been okay but to be fair, I live in a place with only occasional rainfall. We just went through the monsoon season so I had the chance to gather a little more experience with it. (My own pet peeve is that it should always spray at least once before the first auto-activation, to avoid dry-wiping a gritty dusty windshield).

To me, it's evident that the core problem is that the rain-sensing camera(s) (same as the Autopilot forward windshield cameras) are not looking at the large panel windshield that the human driver must look through, and not in the same way optically.

For example, droplets and rivulets close in front of the camera lens have quite a different effect on the image, being largely out of focus. Another way of expressing this is that each focused image point on the sensor is actually the result of many different light-ray paths through the windshield and lens, so the imaged effect of a droplet is mitigated by, and averaged with, image ray paths through adjacent regions of the windshield. This is a well-known effect in photo and video shooting, where you can get a lower contrast but overall a decent image through a remarkably dirty lens or filter.

Now, this saving grace (handling a wet/dirty cover glass) doesn't work if the lens opening is very small, i.e. there isn’t really a wide bundle of rays, i.e. the depth-of-field is very deep - in which case the dirt and water droplets are nearly in focus. I suspect that the Tesla cameras, similar to many popular surveillance cameras, are engineered so that the useful focus depth extends from infinity downto a few feet in front, but the very close weather-exposed surface is deliberately out of focus.

For the human driver, the optical effect is quite different. The windshield surface is two or three feet away and pretty much in focus even when youre watching down the road.. Each droplet thus obscures anything behind it. On the other hand, each droplet blocks only a small piece of the image. On the third hand, there are many more droplets and rivulets across the image field. These differences in the character and severity of the optical obstructions, I think, contribute strongly to the dissatisfaction with Tesla's auto-wiper sensing system.

For any auto-wiper function with a human driver, the trick is to balance the goal of clearing the view against the threshold of being annoying. As discussed above, the Autopilot cameras aren't seeing things the same way as we do, which obviously makes it harder to satisfy the preference of the human driver.

"Aha then", people say, "why doesn't Tesla just use the same well-developed rain sensors as everyone else, instead of trying to save a couple of bucks and annoying us in the process?" I think the answer is built into the observations above: Autopilot sees a rainy windshield differently than the human does, and needs the auto-wiper function to keep its camera view clear. Logically and practically, he best way to do that is to use its own cameras to feed that judgment.

Wishing to override this while in Autopilot is asking for trouble. The human driver can always press the button for extra clearing if he needs to*, but also shouldn't complain that he can't prevent Autopilot from doing the same.

* acknowledging the helpful suggestion of @Daniel in SD in the 10.69 thread, i.e. Tesla's computer should get the hint if the driver repeatedly asks for extra wiper passes. I don't think there's any harm if the wipers do more than AP needs, only if they do less.​

After covering that, what's left is "OK but what about when I'm driving manually? Then I want the system to prioritize my human preference, which was better handled by the sensor I had in my [Mazda|Ford|Audi|whatever]!"

Two points in response to that:

1. Do you, or do you not, wish for the Tesla and its Autopilot cameras to stay active in the background even during manual driving, ready to warn and/or intervene in emergency situations? And especially in the rain? If you do, then we're full circle back to AP camera control of, or at least input to, the wiper clearing function.
2. If not, then do you wish for a completely separate, alternate, wiper control sensor, more tuned to human manual driving needs? This indeed will cost everyone more, and you would surely be asked to acknowledge an agreement on-screen, that you are disabling the ability of the Autopilot to provide emergency monitoring. I think in the end, few drivers would choose that option, so providing a completely separate rain sensor for that subset of drivers and conditions is probably not a great design choice.

I think that if we consider all of the above, and with the imperative of safety over annoyance reduction, this whole drumbeat of auto-wiper dissatisfaction becomes more explainable, and not so clearly a marker of Tesla's ignorance, greed or engineering hubris.

The situation with auto high beams is somewhat similar considering the present equipment, with the main difference being that poor performance can be an annoyance and/or a hazard to other drivers. I won't go into depth on that one, but I do believe that Tesla should have been more proactive in equipping all cars with the Adaptive Matrix headlamps so that a (presumably) really good solution wouldn't be so painful to deploy across the fleet, later on.

 

[AlanSubie4Life]

acknowledging the helpful suggestion of @Daniel in SD in the 10.69 thread, i.e. Tesla's computer should get the hint if the driver repeatedly asks for extra wiper passes. I don't think there's any harm if the wipers do more than AP needs, only if they do less.

He’s also had the excellent suggestion to just see what other cars are doing. Will work in most conditions. Not in some regions of California where all cars are Teslas though.

but I do believe that Tesla should have been more proactive in equipping all cars with the Adaptive Matrix headlamps so that a really good solution won't be so painful to deploy across the fleet, later on.

How does that change anything at all? I guess I understand Matrix headlights if legal could be used to selectively brighten areas without dipping but you still have to figure out whether you should have high beams on or not, and in general you will want to blast the other side of the road with light, if no one is there.

 

[JHCCAZ]

He’s also had the excellent suggestion to just see what other cars are doing. Will work in most conditions.

Indeed. Follow The Herd can work many cases, but don't follow it over a cliff...

However, if you got my argument, it was that auto-wipers need to do what Autopilot needs them to do, not simply what humans want, nor what any other best-in-class auto-wiping vehicles are doing for their drivers. If everyone else is wiping faster though, as I said there's probably no harm in following that. And yes, the California scenario of many Teslas copycatting each other would be a problem, what we call runaway positive feedback, causing furious scrubbing, perhaps mass burnout and a run on replacement wiper motors at Tesla service centers all across the state.
(It could be a Broadway tune, "Any screen you can wipe, I can wipe faster, I can wipe any screen faster than you...")

Regarding headlights, as I said in that last paragraph, I wasn't presenting an in-depth comment regarding the existing Auto high beam function of the existing traditional function headlights.

BTW, a few minutes after I first posted I added a word "(presumably) really good". It doesn't change our present dialogue, but I did want to clarify that any new deployment should be backed by data from successful trials. Too many approaches are promoted or condemned here based on "A is obviously better|worse than B".

To your comment though, I'm not completely sure I understand your position or question:

How does that change anything at all?

If you mean, why did I bring up matrix eadlights when discussing the Auto high beam feature, it's because it's the next major step and could obviate the need for discrete high /low beams at all (read on), and since Tesla has known this for a good while, they could have had hardware ready across most of the fleet, instead of selected models from selected factories.

Of course there would have been cost and supply considerations, but Elon famously made the decision to equip all cars with the Autopilot computer and sensor suite. In contrast, the headlights have been handled quite differently. Perhaps the logic of this will become clear as things develop, but I for one haven't understood it.

So IMO bringing it up was justifiably related to AHB, and to how long users will have to live with the present much-maligned feature, but I'd agree it doesn't directly address complaints about erratic behavior of AHB. As with auto-wipers, my personal experience is that AHB Works reasonably well over the year that I've owned my Tesla. Of the two, I'd say AHB activation probably follows my expectations even better than auto-wipers

you still have to figure out whether you should have high beams on or not

Actually no. With the active-matrix headlights fully functionally implemented, there will no longerbe a distinction between the two. Essentially you'll have a flooded full beam available full-time, with selective dimming (in a good design, not a binary off or on per pixel, but selective illumination intensity) for areas that would otherwise dazzle other drivers, pedestrians or animals, and also to attenuate return from overly reflective signs, avoid washout of safety illumination sources and so on. Essentially, flatten I e. compress the dynamic range of the illuminated scene as well as protect the eyes and cameras of other actors.

The concept of high vs. dipped beams would become a relic of traditional headlight technology, not something to be preserved just because we're used to it.

and in general you will want to blast the other side of the road with light, if no one is there

Exactly, which is essentially what I said above. The issue becomes not "When do I want a matrix-modified high beam", but rather " What exact pattern do I want from my available full-flood beam".

In principle, a fully deployed matrix headlamp doesn't really need a specific "low beam" mode at all. Currently equipped matrix headlight Teslas still have them, I think for two reasons: First, they have to be compatible with currently mandated headlamps, for which the law is written around high and low beam operation, and they have to be compatible with Tesla's harness and canbus. Second, it may be cheaper to stick with a full-time dipped-beam portion without assigning dozens of matrix-selective pixel elements, if one contends that there's no important need to control that portion of the illumination.

Counter to that second point, is that there are already very high resolution full-image matrix headlights that can "paint" stop lines, turn arrows and text onto the near-field road surface, or project cartoons or videos or actual useful information, applications that will surely be exploited both to further advance our safety and to further retard our mental processes.

 

[AlanSubie4Life]

Actually no.

with selective dimming (in a good design, not a binary off or on per pixel, but selective illumination intensity) for areas that would otherwise dazzle other drivers, pedestrians or animals, and also to attenuate return from overly reflective signs, avoid washout of safety illumination sources and so on.

Actually yes. What I was trying to say is that in order to do any of this, you have to detect these things. Note this is the point of failure of today’s system.

So to me it seems it changes nothing. Could there be more awesome responses with matrix high beams given accurate detection? Sure.

bringing it up was justifiably related to AHB, and to how long users will have to live with the present much-maligned feature,

Would not seem to change that timeline.

I'd agree it doesn't directly address complaints about erratic behavior of AHB.

Agreed. It will not change the erratic behavior without sensing improvement. They do not remove the need for sophisticated sensing.

 

[Daniel in SD]

Actually yes. What I was trying to say is that in order to do any of this, you have to detect these things. Note this is the point of failure of today’s system.

No idea how matrix headlights work but it seems like detection would be far easier with them. The detection camera could be out of sync with the strobe of the LEDs and they could simply not shine light on anything that's emitting light.

 

[OxBrew]

My auto wipers wipe too early in some light/rain mist and too late in a deluge (and some other light rain/mist). The too early in the mist (some kinds, not all) is just annoying, but the too late in a deluge is dangerous. Almost never do they wipe correctly. So I don't use auto wipers. And this behavior pretty much negates all the discussion about AP needs to wipe early, because it mostly does not, at least for me (In the PNW we actually get a lot of real rain).

I tentatively retract all my hate on the auto high beams though. Tried them for the first time in several months, on 28.2, and they seem pretty great, at least around town late at night. Never flashed anyone, always dimmed on time. I'm impressed so far. Will need to see how they do on the dark open freeways across Washington State to know for sure. They've really struggled there in the past.

 

[AlanSubie4Life]

No idea how matrix headlights work but it seems like detection would be far easier with them. The detection camera could be out of sync with the strobe of the LEDs and they could simply not shine light on anything that's emitting light.

I guess that is true. I wonder if that is the current actual detection issue though? Is it really not detecting lights because the high beams are on?
Anyway I agree it seems those approaches might help with detection.

I guess it is easier if you just assume everything is a car. Wonder how annoying that would be though? Less annoying I guess.

Would it be a negative to not illuminate street signs though? Seems possibly bad.

Lol, having a rough time with this concept. Of course it would shine light on signs and other dark objects. Seems like it could work well if they can detect lights. Not clear how it would be perceived by the driver though.

 

[Daniel in SD]

I guess that is true. I wonder if that is the current actual detection issue though? Is it really not detecting lights because the high beams are on?
Anyway I agree it seems those approaches might help with detection.

I guess it is easier if you just assume everything is a car. Wonder how annoying that would be though? Less annoying I guess.

Would it be a negative to not illuminate street signs though? Seems possibly bad.

If you do the detection while the camera is off street signs won't be detected as emitting light.
Still haven't found a document on exactly how they're implemented but this data sheet for the LED drivers has a synchronization signal to sync all the lights which makes my hypothesis plausible.

https://www.ti.com/lit/ds/symlink/tps92661-q1.pdf?ts=1666614882469&ref_url=https%253A%252F%252Fwww.google.com%252F

Of course knowing Tesla they probably didn't update the cameras and are planning to do something much more advanced.

 

[AlanSubie4Life]

If you do the detection while the camera is off street signs won't be detected as emitting light.

Yeah I figured that out…eventually.

I guess for the annoying flickering result of all of this they could reduce it by dimming the lights somewhat gradually but not too gradually during the response, maybe.

 

[Boza]

I can understand the frustration of having certain aspects of the car's systems having changed/ ldegraded behavior. Without denying that this is true. nor trying to excuse it, , I think it's perceived as even worse when you've gone through the first learning learning curve and have already adapted to the idiosyncrasies of your new car. Then to have it changed is worse than the experience of a brand-new owner who is learning on that already-changed behavior.

‐------‐----------------------
More thoughts on auto-wiper functionality, and my conclusion that Tesla has probably made the correct choices. For those who are interested, I offer the following line of reasoning:

My experience with auto wipers has been okay but to be fair, I live in a place with only occasional rainfall. We just went through the monsoon season so I had the chance to gather a little more experience with it. (My own pet peeve is that it should always spray at least once before the first auto-activation, to avoid dry-wiping a gritty dusty windshield).

To me, it's evident that the core problem is that the rain-sensing camera(s) (same as the Autopilot forward windshield cameras) are not looking at the large panel windshield that the human driver must look through, and not in the same way optically.

For example, droplets and rivulets close in front of the camera lens have quite a different effect on the image, being largely out of focus. Another way of expressing this is that each focused image point on the sensor is actually the result of many different light-ray paths through the windshield and lens, so the imaged effect of a droplet is mitigated by, and averaged with, image ray paths through adjacent regions of the windshield. This is a well-known effect in photo and video shooting, where you can get a lower contrast but overall a decent image through a remarkably dirty lens or filter.

Now, this saving grace (handling a wet/dirty cover glass) doesn't work if the lens opening is very small, i.e. there isn’t really a wide bundle of rays, i.e. the depth-of-field is very deep - in which case the dirt and water droplets are nearly in focus. I suspect that the Tesla cameras, similar to many popular surveillance cameras, are engineered so that the useful focus depth extends from infinity downto a few feet in front, but the very close weather-exposed surface is deliberately out of focus.

For the human driver, the optical effect is quite different. The windshield surface is two or three feet away and pretty much in focus even when youre watching down the road.. Each droplet thus obscures anything behind it. On the other hand, each droplet blocks only a small piece of the image. On the third hand, there are many more droplets and rivulets across the image field. These differences in the character and severity of the optical obstructions, I think, contribute strongly to the dissatisfaction with Tesla's auto-wiper sensing system.

For any auto-wiper function with a human driver, the trick is to balance the goal of clearing the view against the threshold of being annoying. As discussed above, the Autopilot cameras aren't seeing things the same way as we do, which obviously makes it harder to satisfy the preference of the human driver.

"Aha then", people say, "why doesn't Tesla just use the same well-developed rain sensors as everyone else, instead of trying to save a couple of bucks and annoying us in the process?" I think the answer is built into the observations above: Autopilot sees a rainy windshield differently than the human does, and needs the auto-wiper function to keep its camera view clear. Logically and practically, he best way to do that is to use its own cameras to feed that judgment.

Wishing to override this while in Autopilot is asking for trouble. The human driver can always press the button for extra clearing if he needs to*, but also shouldn't complain that he can't prevent Autopilot from doing the same.

* acknowledging the helpful suggestion of @Daniel in SD in the 10.69 thread, i.e. Tesla's computer should get the hint if the driver repeatedly asks for extra wiper passes. I don't think there's any harm if the wipers do more than AP needs, only if they do less.​

After covering that, what's left is "OK but what about when I'm driving manually? Then I want the system to prioritize my human preference, which was better handled by the sensor I had in my [Mazda|Ford|Audi|whatever]!"

Two points in response to that:

1. Do you, or do you not, wish for the Tesla and its Autopilot cameras to stay active in the background even during manual driving, ready to warn and/or intervene in emergency situations? And especially in the rain? If you do, then we're full circle back to AP camera control of, or at least input to, the wiper clearing function.
2. If not, then do you wish for a completely separate, alternate, wiper control sensor, more tuned to human manual driving needs? This indeed will cost everyone more, and you would surely be asked to acknowledge an agreement on-screen, that you are disabling the ability of the Autopilot to provide emergency monitoring. I think in the end, few drivers would choose that option, so providing a completely separate rain sensor for that subset of drivers and conditions is probably not a great design choice.

I think that if we consider all of the above, and with the imperative of safety over annoyance reduction, this whole drumbeat of auto-wiper dissatisfaction becomes more explainable, and not so clearly a marker of Tesla's ignorance, greed or engineering hubris.

The situation with auto high beams is somewhat similar considering the present equipment, with the main difference being that poor performance can be an annoyance and/or a hazard to other drivers. I won't go into depth on that one, but I do believe that Tesla should have been more proactive in equipping all cars with the Adaptive Matrix headlamps so that a (presumably) really good solution wouldn't be so painful to deploy across the fleet, later on.

I see your logic and agree - and your argument is in the base of the issue. Because of the Tesla Vision pursuit they are crippling basic car functionality. AP with radar (the “old” configuration) + rain sensor would have been a much better experience than Tesla Vision only. Their focus on Tesla Vision is almost religious because it is ignoring facts in the name of a goal that is not clear if it could be achieved.
I am Ok, even supportive, of pursuing Tesla Vision to see if that could work. However, it must be done separately from the rest of the systems that already work. That is why you do not mix “production” and “beta” in critical environments.

 

[mongo]

Getting into the weeds, but phosphorus based white LEDs have a much longer turn off time than direct band gap colors.