Programming for Software Improvements

Well, Elon recently confirmed that the Model S undergoes dozens of physical hardware changes every month. Given that they rely so much on software to do routine hardware things (like even blinking the turn signals), I can imagine what a challenge it is to have a single release that deals with 10,000 or so subtly different cars. Even the processor the software runs on has changed. If I was a betting man, I would bet that the software can't even rely on being able to determine what hardware is in each car.

For instance, the position and receive strength of the key fob antenna has no doubt changed about five times (at least). It is very possible there is no way the car software can test what kind of antenna a given car has. So, somewhere there must be a detailed database of VIN to car components. The database is also probably known (to the programmers) to not be 100% accurate.

On top of all this, I know that the car monitors it's own hardware to anticipate future hardware outages. 12v battery, charging cables, and tire pressure monitors are three that I know the car monitors, I'm sure there are many more.

So I suspect that the car's programming team is, quite simply, overwhelmed. The more cars they sell the more programmers they need to keep up with all the hardware changes they keep making.

I don't think they have come to terms with what this means, although we on the outside certainly see it when we get underwhelming software releases that are way late.

I realize that the autopilot team is no doubt a separate development team, but I suspect the integration of the autopilot software with the rest of the software will be error prone and problematic. In other words, expect big delays for autopilot too.

As far as those trivial software changes, yeah, I agree. I would task one programmer to make them, but again, software integration testing is a bear...

I suspect that they are so overwhelmed with the 'big stuff' that they are just not getting round to the smaller stuff you mention.

Tesla really needs to put some extra effort into the interface development...

2 years in and a calendar and traffic based nav - oh and the location based suspension, are hardly major accomplishments.

Tesla gets a D+ to date.

I really wish they would add shuffle. And Apple lossless support...

It would be interesting to know how their engineering is organized. I wouldn't think the guy who works on the media player would be the same guy who does the interface to the air suspension, but I suppose I could be wrong. As has been previously noted, the calendar is most likely there because Tallulah requested it... you just need her to request shuffle in the media player as well.

That's what you get when you have a company run by one crazy genius!

Also, on a more serious note, anyone who has shipped commercial software products understands that it's one thing to hack something together, another to actually test, integrate, and ship a product, or even a revision to a product. Sure, an engineer could write this quickly, but does Tesla have the QA resources to properly test it? Do they want those QA resources working on shuffle, or something else?

Tesla does have complexity in their software environment, with multiple onboard devices using software/firmware, frequent changes/upgrades to the hardware in production (increasing the number of supported configurations), and localization required to support the car in multiple countries & languages. Anyone of these complexities can make it difficult to deliver software - and the combination only makes it more challenging.

HOWEVER, what Tesla is doing is not unusual - something other companies have been managing. While the hardware in the Model S is innovative and market leading in multiple areas, the software continues to underperform. Tesla seems to be content to provide basic functionality and has not yet met owner expectations of providing updates to bring functionality up to levels available in other vehicles (even to match what's offered in much lower priced or much older cars). Their software testing process still misses major problems, causing start-stops when they distribute releases. And the software releases are distributed in a seemingly random sequence over an extended period. [I've had to go to the local service center twice to get recent software updates - and I still haven't received the promised "annual" map update.]

Tesla can do much better - and should be investing now in making the improvements needed to accelerate bringing new features to market, improving the functionality of their onboard software to at least meet and preferably exceed that of other cars on the market, and to improve the quality of the software distributed to owners.

Adding more people isn't necessarily the solution - it might take a combination of architectural improvements to how their software is designed, an improved software process, coupled with a much better software testing methodology (which could be a combination of increased automated testing plus adding more owners to their beta tester pool) - and possibly some additional software staff (developers and/or testers).

Among Tesla's owners, there are a number of us who have considerable experience in leading and managing complex software products - who would likely be willing to support Tesla in making these changes, or at least could provide significant assistance through more qualified beta testing.

And if they don't fix this soon, Tesla is going to risk increasing customer dissatisfaction, especially when they start ramping up to delivering 500,000 cars per year...

It's (past) time for Tesla to start doing better on their software - and start delivering functionality Tesla has discussed but not yet delivered - such as the App Store, greatly improved navigation, improved browser, 3rd party software (android emulator?) in addition to new features such as improved smartphone (Apple/Android/...) integration and the new autopilot features.

For a company with Silicon Valley ties, shouldn't software be one of the strengths of the Model S??? (Especially since Tesla is nowhere near fully leveraging the potential of the onboard hardware - the continuous Internet connectivity, the large 17" touchscreen display and the software configurable dashboard)

Btw, FWIW, Tesla did already come out with an improved browser in the last few months. It might have been with 6.0. It's just that the new and improved browser still sucks. The speed increased, better compatibility, better touch registration, but still tons of bugs, and still too slow.

While I agree that the Model S software is lacking in many aspects, some simple defaults are just plain silly. In the energy app, having it default to display the instantaneous range estimate, for example, is silly. I almost always, first thing, touch the "Average" button on that display. It would be a simple change to make the default "Average" instead of "Instant", or better yet, make the default the last setting from the previous session. I've seen that request in the fora multiple times.

It occurred to me that making the Model S code "open source", like Linux itself, and having the savvy Tesla community contribute to the development of the code would significantly increase the cadence for improving the software. The major issues with a truly open development environment like that are obviously around vehicle safety, with technical issues related to dealing with the myriad variations of hardware in the wild. I can foresee all sorts of safety, regulatory and legal issues in making that a reality, but they are not insurmountable.

I think Tesla's software team is understaffed and missing some key elements, like quality control and experienced developers (judged by the number and type of bugs each release has contained). They are also very underpaid (I looked), so top talent is going to go elsewhere. Competitive pay needs to get fixed first.

I have a feeling this is the explanation.

yobigd20 said:

I think Tesla's software team is understaffed and missing some key elements, like quality control and experienced developers (judged by the number and type of bugs each release has contained). They are also very underpaid (I looked), so top talent is going to go elsewhere. Competitive pay needs to get fixed first.

Click to expand...

yobigd20 said:

I think Tesla's software team is understaffed and missing some key elements, like quality control and experienced developers (judged by the number and type of bugs each release has contained). They are also very underpaid (I looked), so top talent is going to go elsewhere. Competitive pay needs to get fixed first.

Click to expand...

donv said:

I have a feeling this is the explanation.

Click to expand...

I think some of this situation also extends to the electronic hardware team. The vampire drain when the vehicle is idle could be improved immensely. To give you some idea of the magnitude of the vampire power consumption, assuming the quoted 1% drain per day (Owner's Manual), and my personal experience is a bit higher -- it's roughly 1kWh per day, the power you would consume running a 40W light bulb continuously. For a high technology implementation, there are certain subsystems which need to run all the time, but frankly, 40W is enormous! BTW, a 40W continuous load (~3.3A essentially from the 12V battery which has to be topped off by the DC/DC converter from the HV drive battery) will stress the 12V lead-acid battery too.

My career (before semi-retiring) was developing implantable medical electronics -- pacemakers and the like, which run on 10µW continually for 10 years of more. While there are obviously vast differences in the technologies between pacemakers and automobiles, there are common electronic hardware design techniques that could be used to significantly reduce the vampire power drain and not impact the functionality of the vehicle. Many of these techniques can be implemented in the software as evidenced by the addition of functions like "Stay Connected" in Power Save mode. It takes a holistic view of the intended function, across hardware and software, to engineer a near optimal power implementation. Experience counts.

The slow pace of software development on the Model S has been my biggest disappointment with the car.

I started a thread similar to this one 2 years ago. Sadly, the situation still hasn't changed.

Todd Burch said:

The slow pace of software development on the Model S has been my biggest disappointment with the car.

Click to expand...

I realized early on in my career that I had to take a holistic view of engineering, not separate hardware and software, if I was to arrive at a near optimal overall power solution in a fast-paced development environment. Both disciplines, operating independently, can do things to harm the power profile. Clocking the CPU at too high a frequency wastes power. Not duty cycling the radios wastes power. In a battery powered project (albeit a HUGE battery in the case of the Model S), every engineer (HW and SW) has to think about power, and talk to each other often to make the best decisions quickly. The more time everyone spends up-front with the specification, HW & SW partitioning and design / test verification the better.

I agree on these points above.

I don't need the shuffle in particular, but there are other things. Like access to my iPhone contact faves - and have thousands of contacts - ok, I checked, 2125 at the moment, and I have neither access to favourites, nor a text based function. I know I could use the voice control, but "call peter" is a bit useless when I have 29 Peters in my contacts. Often it is not a "Peter" or a "Mark" I need to call, but a more unique name. Still, I have to browse quite a bit to find it using the screen, while on the phone I just use the search function and type in a few letters.

I also want the energy to stay on "Average" as mentioned above, just make it stay where it was last.

Speaking of energy, if Tesla doesn't make it, I want to make (or have made) a feature that will calculate the actual performance on range, and compare it to distance to destination. It should come up with a number which is simply an estimate of how much range is left when you get to your destination. If not using the built in navigation, I should be able to enter remaining miles, and it will decrease those as I eat up the miles (eh, km). If it would be smart enough, a route in nav would make a simple comparison between current and destination elevation, and add/subtract accordingly to the range estimate. No fun to see 30 miles estimated at destination, if the last part of the trip turns out to be up a mountain (in case one didn't know the area beforehand).

Yes, it would cost Tesla more right now to start making the badly needed improvements to their software process, development staff, and the software itself.

But the longer they wait to do this - the harder it will get - and the longer it will take. And if they don't do something soon about this, the situation could be much more challenging when they're shipping 500K cars per year.

I couldn't agree more with just about everything in this thread.

Why can't I change charging rate via the app?
Turn range mode on / off via app or while charging?
Smarter headlights and day/night mode - learn the drivers habits or just allow adjustable threshold for switching
TPMS by tire

This kinda stuff brings me back to the issue we are all facing, we can't service / modify / work on these cars. Tesla is giving us no information about it and it is a problem.