A flight instructor teaches Tesla Autopilot

[bluetinc]

Papafox,

Thanks for the additional writeup talking to dealing with the automation systems. I've found it very helpful to think about it in the terms you suggested such as monitoring an inexperienced driver, who at times would be fine, and at others you would need be ready to take over, or simply take over. It was worth the wait!

Peter

 

[Papafox]

Papafox,

Thanks for the additional writeup talking to dealing with the automation systems. I've found it very helpful to think about it in the terms you suggested such as monitoring an inexperienced driver, who at times would be fine, and at others you would need be ready to take over, or simply take over. It was worth the wait!

Peter

Bluetinc,
Yes, I think aviation experience is really helpful in understanding safe operations with the Tesla autopilot. The point I most wanted to make is that a person will not likely respond quickly and well enough in a critical situation unless there's already focus on what's going on and what might need to be done. I remember shooting autoland landings in horrible weather as an airline captain. One pilot was monitoring the autopilot and flight path while the other had eyes looking through the windshield, trying to pick up the runway. Because we were getting so close to the ground, there was tremendous focus on the autopilot's performance. If something went wrong (autopilot issue, traffic is allowed into critical area of runway and nav signal gets distorted, or any number of issues), we were ready to take over and command the plane to start climbing again. If you are using autopilot and that semi-truck is crowding your lane, or there's a concrete barrier just a few inches from the side of the lane, or the winding mountain road is making the autopilot work too hard to keep in the lane, you need to be very focused on what's going on and be very prepared to take over. In time, the Tesla autopilot will continue to improve and "critical times" will become fewer and father between, but for now you need to keep your captain's hat on when sitting in the driver's seat. Sometimes, the challenge for Tesla's autopilot is low, and you can relax quite a bit at such times and enjoy the experience. The trick is learning to identify the range from "duck soup" to critical situation and elevate your focus accordingly.

 

[garygid]

p, nicely said.

A Happy (and safe) Thanksgiving to all.
May the world, through some miracle of understanding, find peace and cooperation.

 

[jscholl]

Papafox, thanks for the awesome instruction! I don't actually have a Tesla, but I have a couple questions anyways, if that's okay.

1) How much attention should one pay to the sensors? I know pilots of small planes generally conduct a walk-around, though I never figured out what kind of problems that can detect, so I have no idea if that would apply. If the radar, ultrasonic sensors, or camera are clogged, covered, or otherwise obstructed, would you want to know that before you got in the car, or would you trust the AutoPilot to disable itself?

2) Would you recommend testing and/or familiarizing yourself with the lane departure warning, blind spot warning, or collision warning systems? How about the lane detection and ultrasonic sensors? I know nothing about flight instruction, but I was kinda surprised that your suggested first session was to just jump in and turn on both TACC and AutoSteer.

 

[Papafox]

Papafox, thanks for the awesome instruction! I don't actually have a Tesla, but I have a couple questions anyways, if that's okay.


1) How much attention should one pay to the sensors? I know pilots of small planes generally conduct a walk-around, though I never figured out what kind of problems that can detect, so I have no idea if that would apply. If the radar, ultrasonic sensors, or camera are clogged, covered, or otherwise obstructed, would you want to know that before you got in the car, or would you trust the autopilot to disable itself?


2) Would you recommend testing and/or familiarizing yourself with the lane departure warning, blind spot warning, or collision warning systems? How about the lane detection and ultrasonic sensors? I know nothing about flight instruction, but I was kinda surprised that your suggested first session was to just jump in and turn on both TACC and AutoSteer.

Jscholl, I have heard reports of the large radar sensor in front degraded because of an excess of large squished bugs or other substantial debris on the sensor, but I have not experienced this issue myself. I'm thinking the other sensors on the car may not be particularly inclined to degradation because since they are mounted at an angle to the direction of movement, any strikes of big, juicy bugs will be an angle hit and most of the bug is likely to continue to move beyond the sensor. Additional input welcome.

Any sensor is capable of failing completely on its own or experiencing degraded performance because of debris. however, and caution is needed to avoid depending entirely on sensors until some sensor redundancy exists.


1) As far as preflight (predrive?) inspection is concerned, I have one preflight action I always perform: checking to make sure that my cup of coffee is not on top of the Tesla and is instead place inside in a cup holder. Let's move to safety-related checks, though.

I make a point of having a spray bottle of detailing solution in the trunk and I use a microfiber towel to remove accumulations of debris from sensors, including the large radar sensor. I've never let the debris build up to the point where I see degraded performance, and I invite other perspectives here.

Let's not forget about the forward-looking camera behind the rearview mirror, too. This camera is used by the lane-warning system and other parts of autopilot, and for this reason a clean windshield will most likely enhance the functioning of the autopilot. Does just the area of the windshield near the rearview mirror or does a larger portion of the windshield need to be clean in order to promote optimal camera behavior? I'm inclined to believe a larger area is needed because the width of the camera coverage might be wider than you imagine. If you own a dashcam, you know that quite a large area of clean windshield enhances your dashcam picture, and I would imagine that the image used by autopilot would be similarly improved by a clean windshield. Bottom line: you'd be wise to clean the windshield glass prior to the next leg of your journey. I have never let the windshield get covered with bugs or other debris while the AutoSteer function is used, and for this reason my words here are speculation, rather than hard facts. Maybe someone can comment who has seen AutoSteer degradation when the windshield is heavily mucked up.


2) There's much to be said regarding this second set of questions. Let us discuss them with individual replies.

 

[Papafox]

In looking at lane departure warning, blind spot warning, and collision warning, so much of the secret of using these features well is to understand just how they function. Let's take them one at a time.


Remember that Tesla's manual on these features can be brought up on the touchscreen through the following steps: Controls > Settings > Manual (in lower left-hand corner) > Owner's Manual > Driver Assistance.


Lane Departure Warning
This feature is available in appropriately-equipped Tesla vehicles when the Driver Assistance (Autopilot) option is enabled. A forward-facing camera near the rear-view mirror of the Tesla monitors the vehicles position relative to the lane markers. When the vehicle gets too near or crosses a lane edge, the lane departure warning will provide a simulated rumble-strip vibration to the steering wheel. Note that since the forward-facing camera is used for this feature, the camera must be able to see through the windshield, and reasonably-discernible lane markers must be visible to the camera. Poor lane markings or lane markings hidden by rain and glare or by snow would likely not activate the lane departure warning.

 

[Papafox]

Blind Spot Warning
About the time Tesla introduced tall headrests on the rear seats to increase the safety of those occupants, the company introduced a blind spot warning system. You can see why there was a connection here because the tall headrests cut off the view through the corners of the rear window and effectively widened the size of the blind spot (that area that can’t be seen through the rearview mirror or a casual glance to the side of the car). When I took a test drive in a 70D, the Tesla sales lady said, “Don’t worry about the higher seat backs because this car has a blind spot warning system." Sure enough, as a car passed through the blind spot off our left or right side, a red line appeared on the Model S image just below the speedometer indication.

The high headrests effectively narrow the viewing angle out the rear window


I never gravitated towards using this system, however, and here’s why. When you are changing lanes, you need a fast method of clearing for traffic and then beginning the lane change. First you evaluate through the rearview mirror whether there’s someone in your future lane who is behind you and moving forward quickly enough to make that lane change a bad idea, then you check your sideview mirror, and finally you look out that side of your car to see if it is clear, and then you begin the lane change. If this process takes too long, some hotshot behind you can zip forward and threaten the maneuver.


Pilots have a name for the manner in which they shift their eyes around the instrument panel. They call it “scan pattern”. A good instrument scan needs to be thorough enough but fairly quick, and the same rule applies to scanning for traffic before you change lanes in your Tesla (fast but through). When I change lanes, I check the rearview mirror to assess threats from behind, I next scan my side mirror to evaluate the blind spot, I take a look out the side window to see if a vehicle is beside me, and then I signal and change lanes. It’s a three step process and my eyes are moving either to the left at each step for a left lane change or to the right in two steps for a right lane change. Now, if you wished to check for an intruder in your right-side blind spot by using the Tesla’s blind spot warning system of that time, your eyes would have to move from your rearview mirror, left to the instrument panel in front of you (and refocus because you’re looking at something closer), then right to your right side mirror (and refocus again), and then out the right side of your car. Can you see how less efficient that scan pattern is? Thus, even when it was available, I never used the Tesla blind spot visual warnings on the instrument panel. If the blind spot indicator was located on the side mirror as it is in some other cars, I probably would have used it, though.


Notice that I said I used my side mirrors to check the blind spots. I can do this because my mirrors point outward at a greater angle than where most people set their mirrors. Some experts suggest setting your side mirrors to cover your blind spots instead of duplicating the view you already have through your rearview mirror. Next time you’re on the highway, try adjusting one side mirror and then the other so that a car coming up behind you is always visible in the adjacent lane through either your rearview mirror or side mirror, but the overlap is minimal. The more you reduce the overlap between what can be seen through the two types of mirrors, the more the side mirrors can point outward and cover a larger portion of the blind spot. I’m very happy with the setup of my mirrors now and I can watch a car move from the rearview mirror edge into my side view mirror and about the time it disappears from the side view mirror I can see it off the side of my car.


Caution #1: Setting your mirrors in a radically new fashion will take some getting used to. Be very careful if you try this new mirror-setting technique.


Caution #2: No mirror setup is completely foolproof. For example a small motorcycle coming into your blind spot from two lanes over and moving within the blind spot might elude your scan. It is best to supplement your mirror views with “situational awareness”, in which you have a good feel for what is happening trafficwise around you. If your mirrorwork is less than perfect and you’re thinking “what happened to that motorcycle?”, you may take enough time to find him before signalling for a lane change. Also, remember that your autopilot-equipped Tesla can scan your blind spot and other traffic areas too. When trying a new mirror strategy, you could likely benefit from having your Tesla drive itself and double-check your decisions.


When I drive my AutoPilot-equipped Model S these days, I no longer see red lines alongside the auto image on the instrument panel. I see the typical semi-circles radiating out from the front or rear left or right sensors, but I don’t see the lines any more. Personally, I would rather drive with the AutoPilot on including AutoSteer and back it up with my blind spot scans through the side mirrors than rely upon a momentary illumination on the instrument panel.

 

[Papafox]

Side Collision Warning and Steering Intervention
We already know that a Tesla equipped with driver assistance components can alert the driver if the car approaches or crosses clearly-marked lane markers on a major roadway. Simulated rumble-strip noise and steering wheel vibrations are used to alert the driver. Other features associated with lane assistance exist as well, and one of these additional features may be a surprise to anyone who hasn’t carefully read the Tesla manual.


Side Collision Warning
If an object is detected to the left or right side of the Tesla, including the blind spots, semi-circles of white, yellow, orange or red lines radiate from the appropriate sensor location on the dashboard’s image of the Tesla (see photo below).

No surprises here: the white arc shows farthest objects that are detected, yellow is the next level of closeness to your vehicle, then comes orange, and finally comes red when the object (another car, a concrete barrier, etc.) is deemed so close to be a major concern. When a red arc appears, a chime will sound.


Steering Intervention
Here’s a surprise to many Tesla owners. When your appropriately-equipped Tesla is either close to the edge or your lane or actually crossing into the adjacent lane and sensors pick up a nearby object on that side of the Tesla, steering intervention may kick in to prevent a collision. Tesla says that this feature only works when you are on a roadway with clearly marked lanes. Steering intervention provides steering input that directs the Tesla either back into its lane or laterally within the lane in order to prevent a collision. Note that steering intervention is not activated when AutoSteer is active (presumably AutoSteer has its own mechanisms for preventing this type of collision). A message on the dashboard illuminates whenever steering intervention has activated. A variety of conditions, including speed between 30 mph and 85 mph, are necessary for steering intervention to step in.


Searching through forum posts, I have been unable to find examples that speak of personal experiences with steering intervention. One post speaks of the driver’s Tesla moving over to the far side of the lane as a motorcycle passed along a lane marker, but no mention was made about whether the Tesla was on AutoSteer at the time.

 

[Papafox]

With the release of version 7.1 software, modifications to the first post of this thread will be in order. Looking forward to experiencing and hearing about the following enhancements to Autosteer:
* Autosteer has been improved to keep Model S in its current lane when passing highway exits
* Autosteer has been improved to keep Model S in its current lane when lane markings are faded
Your comments are welcome, as usual

 

[jbcarioca]

With the release of version 7.1 software, modifications to the first post of this thread will be in order. Looking forward to experiencing and hearing about the following enhancements to Autosteer:
* Autosteer has been improved to keep Model S in its current lane when passing highway exits
* Autosteer has been improved to keep Model S in its current lane when lane markings are faded
Your comments are welcome, as usual

I drove roughly 300 miles mmediately following the 7.1 upgrade.
- Autosteer now is indeed far more stable when passing exits with no continuation markings. I tested that by driving in the exit lane as much as possible on I 75 from Sarasota to Fort Lauderdale FL. Almost always the passage was stable without even the previously typical 'twitch'. However, when the exit is on a bend in the road placing the car on the outside of the bend there is still some 'twitch' as there is if a vehicle immediately ahead takes the exit.
- Autosteer is significantly improved with faded markings, but also is less confused by construction zones. Two construction zonas I frequently pass have old faded markings with brighter temporary ones. Further there are several bends within the construction zones that conflict with older lane markings. Prior to 7.1 Autosteer immediately demande imediato takeover. Post 7.1 each of these was handled without problem by autosteer.
- another issue mentioned in other threads is the imposition of forced "steering wheel holding" every five minutes or so. I find that to be generally true regardless of road conditions, but is more frequent in rain, with inconsistent markings or in glare. Personally I am quite pleased with the revised alert because they do act to increase driver vigilance in potentially difficult situations. Previously an inattentive driver might have been at higher risk of sterring error IMHO.

 

[Papafox]

jbcarioca, thanks for the report! I just received 7.1 and will be driving extensively over the next few days to add additional perspective and rewrite the first post. I'm currently working on a post about "summon".

 

[Papafox]

A 7.1 Autopilot User's Frame of Mind

While driving 850 miles this past weekend with autopilot version 7.1, I enjoyed the very significant improvements in Autosteer. Wow! The product is very impressive, and herein lies a couple gotchas.

Lots of changes have been made in 7.1. A good software engineer and a pilot would both regard the situation similarly, i.e. changes (as good as they are) introduce potential new problems. Thus, stay on your toes.. Here's an example. I've noticed that if a concrete barrier is located just a few inches to the left side of the left lane's edge markings, the Tesla will now favor the right side of this lane. This is similar to the way I would drive. Just how good is the Tesla Autosteer at dealing with vehicles crowding the right side of your lane at such times? Chances are the Tesla will handle the situation well, but you need to find out. And how about if you throw in faded lane markings and a winding mountain road? Remember that Autopilot challenges tend to be additive until you reach a point where the performance is not good enough.

Here's another gotcha. Some users will find that 7.1 Autopilot performance is so good on well-marked highways that they will start treating the autopilot as an autonomous driving mode, rather than the beta product that it still is. This is a classic overconfidence problem.

For drivers who sat out Autopilot 7.0

Some Tesla owners don't want to be beta testers, particularly with the first release of a new product. Your position is reasonable. Nonetheless, consider the safety benefits that are available to you in Autopilot 7.1.On highways with well-marked lanes and curves that are not too sharp, Autopilot 7.1 does a nice job of keeping you at a safe position in your lane (with a few exceptions), and the Traffic Aware Cruise Control (TACC) does a magnificent job and keeping you at a good distance from the vehicle in front of you. If you need to turn your attention away from the windshield, even for a short period of time, you're safer with Autopilot turned on than without it.

Weather driving is another reason why Autopilot 7.1 can improve the safety of your drive. I had the opportunity to navigate through much light-to-heavy rain on my trip, and Autopilot 7.1 made the trip easier, safer, and far less stressful. I discovered that TACC recognized speed changes in the car ahead of me quicker than I could, and thus the Traffic Aware Cruise Control prevented the kind of quick braking that could cause someone behind me to get uncomfortably close to hitting my Tesla's tail end. You want to use TACC in bad weather, even if you're not yet a fan of Autosteer. As for Autosteer, combinations of poor lane markings, wet roads, glare, and precipitation can render Autosteer unavailable or unreliable. How do you know if Autosteer is unreliable? One clue is that the blue lane markings on each side of the Tesla graphic below the speed indication start to become intermittent. Another clue is that the Tesla's lane-keeping performance does not strike you as good enough. In such a case, you're better off disconnecting Autosteer but keeping TACC active (turn the steering wheel slightly to turn off Autosteer but keep cruise control). Sometimes, though, even at night, the quality of lighting and lane markings allow Autosteer to function well, even in heavier rain. My feeling is that if Autosteer is doing a good job, leave it on. You are then freed up to look at the big picture and perform tasks such as changing environmental controls to keep all your windows free of fogging. Try the Autopilot, you will likely be pleasantly surprised, but use the suggestions on the first post of this thread, please, as you explore this new technology.

 

[Papafox]

The New Copilot Analogy

In the Autopilot 7.0 days before the release of our current version of the software, I likened your relationship to the Tesla Autopilot as a crew situation in which you are the captain and the Autopilot is your copilot. I described the 7.0 Autopilot as being like a very green new copilot. This copilot is very focused but cannot be relied upon without supervision. The copilot can reduce your workload when conditions are good, but you needed to keep your hands on the wheel, you needed to keep the traffic situation in mind, and you needed to take over any time the copilot (Autopilot 7.0) pushed your comfort level too much.

With Autopilot 7.1 you are still very much the captain and your Autopilot is very much your copilot (one who also requires supervision). Nonetheless, you realize that this new copilot can actually do certain things better than you. Traffic Aware Cruise Control can hold its position better behind the car in front of you than you can now. For this reason, you'd be nuts not to take advantage of TACC at times when you're working hard (bad weather, heavy traffic, fatigue, etc.). Autosteer in 7.1 may possibly also do a better job than you at keeping the best position in your lane when the road markings and conditions are favorable (with a few exceptions). Your realize now that Autopilot 7.1 will grow up to be a captain some day. Until then, though, you remain in charge and don't let your mind wander too far from the task of driving. There were times with Autopilot 7.0 when you felt you were babysitting the copilot. Now, with 7.1, you truly can share the tasks of driving as more of a crew effort.

 

[bmah]

Here's a comment I'm tossing out there for, well, other comments:

A big difference I noticed when going to 7.1 is that when I engage the TACC, it's initially set to a higher target speed than it would have on earlier software (7.0 and before). On 7.0, I think the initial speed was your current speed, or maybe the last target speed you used with TACC. With 7.1 the default speed appears to be the speed limit plus your speed limit warning delta (so for example if I'm driving where the speed limit is 65 mph and my delta is +10, my initial TACC setting is 75). I believe this is documented in the release notes.

On the freeways I travel, I'm often in a situation where the flow of traffic is well below the speed limit. For example this morning, I enabled autopilot on a congested freeway when traffic was moving about 25 mph, but the system gave me an initial TACC speed setting of 75 mph. I am very uncomfortable having a large difference between my current speed and the TACC setting (I define "large" as more than 15 mph when in a steady state). In this situation, TACC would keep me at the flow of traffic, but it could lead, for example, to some surprising acceleration if, say, the car in front of me changed out of my lane and TACC perceived a large gap between me and the next car in front. (This isn't intrinsically unsafe, but having the TACC suddenly speed up sure grabs your attention and it can be very unsettling to passengers.)

So one of the first things I usually do after turning on autopilot (rather, TACC with or without autosteer) is to decrease the target speed to something close to my current speed if there's too big a difference between those two speeds. In particular my personal rule is a target speed of no more than 10-15 mph over my current speed in congested and/or heavy traffic. This lets my copilot (er, TACC) accelerate to close gaps opportunistically while still putting a limit on how much faster it can go without my explicit permission.

 

[Papafox]

Bmah,
Thanks for pointing out this change and showing how you deal with it. It's been a few weeks since I was back on the mainland driving my autopilot-equipped Tesla, but as I remember, the 7.1 TACC will choose the higher of the three speeds:
* Current speed
* Last set speed IF car HAS NOT been parked since that last instance of using TACC
* Speed limit plus user-set margin IF car HAS been parked since last instance of using TACC

I think the value of this approach is that traffic ahead of you should normally keep you from accelerating quickly if traffic is slow. You then have the opportunity to reset your speed, as you suggest. On the other hand, if you are cruising right along with no traffic immediately ahead, you likely don't want TACC to slow you down when it is engaged. Of course you can set a slower speed at any time, but for that moment right when the TACC is engaged, you want a minimum of surprises.

 

[Beryl]

In this situation, TACC would keep me at the flow of traffic, but it could lead, for example, to some surprising acceleration if, say, the car in front of me changed out of my lane and TACC perceived a large gap between me and the next car in front. (This isn't intrinsically unsafe, but having the TACC suddenly speed up sure grabs your attention and it can be very unsettling to passengers.)

It appears to me that TACC 7.1 (2.10.56) deliberately maintains a slow speed until it can judge the distance of any vehicle in front of the one exiting the lane. In fact, the speed stays so low that often I need to manually accelerate to avoid irritating drivers behind me. If there is no one behind me, I will let TACC work alone but a rapid increase in speed has never occurred.

 

[Frisco-Dad]

@Papafox thanks for the great article. I ordered a S85 inventory car and I'm learning as much as possible before the car is shipped to my local service center for delivery .

 

[wcalvin]

Apropos the comment about the funny feelings as the driver is made a passenger by TACC and AutoSteer:

As driver, your brain is initiating the moves. The movement centers warn the sensory path to expect the consequent input. Mismatches are flagged as important.

All of this is lost when you become a passenger, no longer initiating the actions. However, one can partially compensate by mentally driving, as my wife does from the passenger seat to avoid motion sickness.

 

[Boatguy]

When I first test drove the AP (my car is on order), when changing lanes I was told to flip on the turn indicator to effect the lane change and I was annoyed that I had to manually cancel the turn indicator once the lane change was completed. In my second test drive I used the momentary contact position, holding the stalk until the lane change was completed, then releasing. That worked fine also.

Depending on the situation and driver, one or the other may be more convenient.

 

[Papafox]

@Papafox thanks for the great article. I ordered a S85 inventory car and I'm learning as much as possible before the car is shipped to my local service center for delivery .

Frisco-Dad, first of all, congratulations! When I wrote the first post on this thread, it was aimed at helping current Tesla drivers transition to the lane-holding capabilities of the autopilot. If you have no experience with either the Traffic Aware Cruise Control (TACC) or the lane-holding functions, my suggestion would be to first become very comfortable with the TACC before jumping into the full autopilot with lane-holding (autosteer). The reason I say this is that the TACC is extremely solid technology, and it's the best place to get your feet wet with this technology. In fact, I've discovered that TACC does a better job than I do at detecting speed changes in the car ahead of me. It's a fabulous tool for use in long drives, especially when you get a bit tired. The lane holding capabilities of the autopilot have improved significantly in the move from ver 7.0 to 7.1, but there are still situations when the technology can be compromised, usually in a construction zone or in areas where the lane markers are deficient. I think if you use both at the same time as you learn the Tesla, most of your focus will out of necessity be on lane holding and you won't get a chance to learn the fine points of TACC as quickly compared to if you concentrate on it first.

I'm going to revise the first post at some point to make it more applicable to people such as yourself who are coming to the Tesla without a long use of TACC under your belt.