New Consumer Reports online article about Model S citing problems in their sample

That's the difference between Consumer Reports and a lot of the other media (*ahem* Edmunds *ahem*). Even when Consumer Reports has the anecdotal evidence, they clearly point out it is anecdotal and can't be relied on as representative of the car model in general. The other media won't point that out because the only thing they have is anecdotal evidence and do not have a means to reliability gather data to get a statistically significant sample size.

Also important to recognize that both they and Edmunds had early cars, which many of us know had...well...more than their share of issues, the vast majority of which have since been resolved.

I clicked the link and went right to the article without a paywall. The problems seem minor, such as:

"Just before the car went in for its annual service, at a little over 12,000 miles, the center screen went blank, eliminating access to just about every function of the car, including popping open the charge port. The shop, a newly opened service center in Milford, Conn., performed a “hard reset” that restored the car’s functions."

That easily fixed yourself. I've never owned a tablet that didn't freeze. Sure, the screen on my Tahoe Hybrid has never froze but it's tiny and doesn't do much.

"Then at about 15,700 miles, we found that the front trunk lid wasn’t responding to the release, which is a virtual button on the central screen. "

Who uses the screen to pop the frunk? But in fairness it seems to be a latch problem and not a screen problem so it seem odd that they first seem to attribute it to the screen or software.

"We also had the Tesla-supplied adapter for non-Tesla EV chargers come apart. "

The J1772? I thought it was one solid piece.

"Again, the car went in to the service center for two days and got its front trunk latch replaced, a new charging adapter was thrown in, and the latest firmware 5.12 was downloaded. Unlike other ones, this update actually was not sent over the air. "

Wrong since all updates can be done over the air but they can also be done manually at the service centers. They give the false impression that for certain updates you have to bring in the car for service. [Edit: I am wrong. 5.12 had to be done at service centers.]

"Based on last year’s big auto-reliability survey, we gave the Tesla Model S a score of average, based on input from 637 owners of 2012 and 2013 models. By September, Consumer Reports will be analyzing this year’s reliability survey, which will also include the 2014 models. It will be interesting to see how the Model S will score after we tabulate the new data. Given the number of bits and pieces Tesla has replaced on our car, it might be tempting to guess that its reliability score will go down. The reality is, it might—depending on the frequency and severity of problems reported by our subscribers and whether they show that reliability is below average. Bear in mind that the experiences with our test cars are purely anecdotal and never factor into our reliability ratings. After all, it's a sample size of one. Along with the rest of the motoring world, we anxiously await the conclusions of our latest reliability analysis due this fall."

I think the only other problems were the handles not retracting properly and a faulty latch on the third row seats. I guess all those problems do all add up but nothing seems too significant. I can post the whole article if that's allowed and if others don't have access to it. Just let me know.

Canuck said:

"Again, the car went in to the service center for two days and got its front trunk latch replaced, a new charging adapter was thrown in, and the latest firmware 5.12 was downloaded. Unlike other ones, this update actually was not sent over the air. "

Wrong since all updates can be done over the air but they can also be done manually at the service centers. They give the false impression that for certain updates you have to bring in the car for service.

Click to expand...

Wasn't 5.12 a service-centre-only update (ie; not released over the air)?

Canuck said:

I can post the whole article if that's allowed and if others don't have access to it. Just let me know.

Click to expand...

The direct link seems to work, I.e. no paywall. Respecting others copyrights we don't allow posting of full articles (quoting snippets is fair use).

stopcrazypp said:

That's the difference between Consumer Reports and a lot of the other media (*ahem* Edmunds *ahem*). Even when Consumer Reports has the anecdotal evidence, they clearly point out it is anecdotal and can't be relied on as representative of the car model in general. The other media won't point that out because the only thing they have is anecdotal evidence and do not have a means to reliability gather data to get a statistically significant sample size.

Click to expand...

Todd Burch said:

Also important to recognize that both they and Edmunds had early cars, which many of us know had...well...more than their share of issues, the vast majority of which have since been resolved.

Click to expand...

Canuck said:

I clicked the link and went right to the article without a paywall. The problems seem minor, such as:

"Just before the car went in for its annual service, at a little over 12,000 miles, the center screen went blank, eliminating access to just about every function of the car, including popping open the charge port. The shop, a newly opened service center in Milford, Conn., performed a “hard reset” that restored the car’s functions."

That easily fixed yourself. I've never owned a tablet that didn't freeze. Sure, the screen on my Tahoe Hybrid has never froze but it's tiny and doesn't do much.

"Then at about 15,700 miles, we found that the front trunk lid wasn’t responding to the release, which is a virtual button on the central screen. "

Who uses the screen to pop the frunk? But in fairness it seems to be a latch problem and not a screen problem so it seem odd that they first seem to attribute it to the screen or software.

"We also had the Tesla-supplied adapter for non-Tesla EV chargers come apart. "

The J1772? I thought it was one solid piece.

"Again, the car went in to the service center for two days and got its front trunk latch replaced, a new charging adapter was thrown in, and the latest firmware 5.12 was downloaded. Unlike other ones, this update actually was not sent over the air. "

Wrong since all updates can be done over the air but they can also be done manually at the service centers. They give the false impression that for certain updates you have to bring in the car for service.

"Based on last year’s big auto-reliability survey, we gave the Tesla Model S a score of average, based on input from 637 owners of 2012 and 2013 models. By September, Consumer Reports will be analyzing this year’s reliability survey, which will also include the 2014 models. It will be interesting to see how the Model S will score after we tabulate the new data. Given the number of bits and pieces Tesla has replaced on our car, it might be tempting to guess that its reliability score will go down. The reality is, it might—depending on the frequency and severity of problems reported by our subscribers and whether they show that reliability is below average. Bear in mind that the experiences with our test cars are purely anecdotal and never factor into our reliability ratings. After all, it's a sample size of one. Along with the rest of the motoring world, we anxiously await the conclusions of our latest reliability analysis due this fall."

I think the only other problems were the handles not retracting properly and a faulty latch on the third row seats. I guess all those problems do all add up but nothing seems too significant. I can post the whole article if that's allowed and if others don't have access to it. Just let me know.

Click to expand...

It might wind up being more reassuring, an early car had several minor problems, that were fixed in a far superior manner to existing service deptartments.

I think this will be a huge deal if the reliability drops below average in the annual review

Most likely it will drop. Not because of the minor things mentioned in that article but due to drivetrain related issues.

But I don't think it would impact sales though.

I think losing consumer reports recommendation would be a big hit to sales

I'm hoping these reports help Tesla improve. I know as "earlier" adopters we tend to give Tesla more slack, but I know when I recommend the car, I tell people honestly, that the car is amazing, but they are still improving on other areas. It's a double edge sword and I hope the service can keep up with these "minor" issues, as I know it takes a long time to schedule any appointment out here in CA. So people just build up issues until they have to go in. For the amount of money spent, I'd prefer not to have gone in for service 5 times in the last 4 months. I just like the car so much I've put up with it. For other people, they wont be so kind.

Todd Burch said:

Also important to recognize that both they and Edmunds had early cars, which many of us know had...well...more than their share of issues, the vast majority of which have since been resolved.

Click to expand...

That may depend on how you define "early". Consumer Reports' last survey showed the 2012's being very good, in fact much better than the 2013's; perhaps being "personally inspected by Elon" and shipped slowly had a real effect.

But I do agree that early 2013 cars likely had more problems than new ones do, and that not many were shipped in 2012. EDIT: it looks like about 3200-3300, thanks Dave.

ChadS said:

That may depend on how you define "early". Consumer Reports' last survey showed the 2012's being very good, in fact much better than the 2013's; perhaps being "personally inspected by Elon" and shipped slowly had a real effect.

But I do agree that early 2013 cars likely had more problems than new ones do, and that not many were shipped in 2012. In fact 2012 was pretty much just the Signatures, I think? (I don't think CR or Edmunds had one THAT early).

Click to expand...

I haven't seen CR's questionnaire on reliability, but I have to wonder how prone to bias it might be--if you get a small cohort of enthusiastic early owners, e.g.. I am not saying anyone hid anything intentionally, just observing that a small group of early supporters might be more willing to overlook small issues that later, more mainstream owners may be more prone to report.

On a personal level, I really hope they don't pull their "recommended" rating just because I don't want to have to listen to it from my in-laws.

ChadS said:

In fact 2012 was pretty much just the Signatures, I think? (I don't think CR or Edmunds had one THAT early).

Click to expand...

My car was probably one of the last 2012's. It is VIN 3236 and has a build date of 12/31/2012.

Canuck said:

"Again, the car went in to the service center for two days and got its front trunk latch replaced, a new charging adapter was thrown in, and the latest firmware 5.12 was downloaded. Unlike other ones, this update actually was not sent over the air. "

Wrong since all updates can be done over the air but they can also be done manually at the service centers. They give the false impression that for certain updates you have to bring in the car for service.

Click to expand...

I didn't read it that way at all. Everyone I know who received 5.12 got it at the service center, not over the air. Regardless, it's difficult to draw the conclusion you are drawing from CR's words.

AmpedRealtor said:

I didn't read it that way at all. Everyone I know who received 5.12 got it at the service center, not over the air. Regardless, it's difficult to draw the conclusion you are drawing from CR's words.

Click to expand...

That's news to me. Correction noted and I edited the original post.

JST said:

I haven't seen CR's questionnaire on reliability, but I have to wonder how prone to bias it might be--if you get a small cohort of enthusiastic early owners, e.g.. I am not saying anyone hid anything intentionally, just observing that a small group of early supporters might be more willing to overlook small issues that later, more mainstream owners may be more prone to report.

Click to expand...

For many years, some owners of cars that score poorly have assumed (not just questioned!) that it is biased. I haven't been a subscriber for a couple of years, but the last time I filled out one of their auto reliability questionnaires, it looked pretty good. As I recall the questions were very straightforward, something like "have you had any troubles in area X that required service". It wasn't leading, should be the same for each brand, and not likely to be accidentally misunderstood only by certain types of people. (Though there could be problems with assigning some problems to the correct category).

Of course, it is possible that owners of some brands lie (but which way?) more than owners of other brands, but I don't see that as the likely case. And while CR subscribers (and more to the point, survey respondents) are no doubt not fully representative of the population at large and I have no problem believing that CR's failure percentages are not exact, I find it hard to believe they would be skewed in such a way that answers to these questions would significantly favor some brands over others. As long as they've got enough answers to be statistically significant, their survey seems far better than any other source of data out there. Only Tesla has better data, and they aren't sharing. (Although I have heard that from their financial reports, their holdback reserve for warranty work sounds great. I haven't seen the numbers myself).

We can't guarantee CR's results are perfectly representative, but until a better (or even reasonably similar) source of data for reliability comes out with different results, I think it makes the most sense to assume their numbers are likely close. Certainly it makes no sense to dismiss their results, though many do because they don't like them.

I think the biggest problem is that our brains handle emotional input (OMG! I just read three horror stories from owners!) far better than numeric input (less than 0.6% of owners report a problem with x). We tend to give more weight to the horror stories and assume they are representative. That's why more people are afraid of flying than driving, even though driving is far more dangerous.

New video CR just posted outlining the previous article

ChadS said:

I think the biggest problem is that our brains handle emotional input (OMG! I just read three horror stories from owners!) far better than numeric input (less than 0.6% of owners report a problem with x). We tend to give more weight to the horror stories and assume they are representative. That's why more people are afraid of flying than driving, even though driving is far more dangerous.

Click to expand...

Flying also has the key issue of control and a lesser issue of opacity (you can't see what the crew are doing). Not too many people mention being afraid of riding a train or bus, though.

Maybe it's the concern that your death could be the result of falling from a great height. It's not a very nice idea.

PS I'm not afraid of flying, though I admit I get slightly nervous at take-off.