VehiclesFashionRecipesBlogsHuntTravelsSportFunHandmadeITEducation
Mini-Games
x

x
zakruti.com » Auto & Vehicles » Alex on Autos
The Problem With Reliability Scores Is Getting Worse - The Hard Truth About Consumer Reports Data

The Problem With Reliability Scores Is Getting Worse - The Hard Truth About Consumer Reports Data

FBTwitterReddit

video description

Rating: 4.0; Vote: 1
The real problem with Consumer Reports' data isn't really CR's fault, but the data is getting sketchy anyway. Why Because the number of people reporting scores is shrinking rapidly, giving the dataset more holes than Swiss cheese. What can be done about it Let's talk.
Date: 2025-07-08

Comments and reviews: 20


As someone who grew up in Germany and now lives in Canada, I have often wondered myself why the perceptions of reliability for brands are so different in North America and Europe. As Alec mentioned, the German brands get touted as incredibly reliable there, and I don’t know how often I heard of VW Golfs that just ran and ran and ran or how this or that Mercedes never needed anything but the regular maintenance in Germany, while here these brands get just average reliability ratings or worse.
And then I came across a video about a Ram 1500 that was getting its 100k mile checkup recently - and the thing had nothing wrong with it, everything was in great condition. In the video this was used as an example to point out that if you just follow the maintenance schedule, then these cars are perfectly reliable and unproblematic.
So basically, the difference in perceived reliability between here and Europe might be differing attitudes to maintenance. In Europe people keep up with maintenance without question. In Germany specifically, each and every single car has to undergo a thorough inspection that checks for street worthiness but also general safety and environmental impact every couple of years. That means that the cars on the roads in Germany are in a much better condition than here. It’s quite amazing how many cars you see on North American roads that are literally rusting away under the driver’s ass. So, reallyno wonder reliability differs between here and Europe.
Which actually makes me wonderdo Consumer Reports (and other reliability surveys) take maintenance or lack of maintenance into account at all Do they check whether the owners follow the recommended maintenance intervals I’d expect that that could make a huge difference for the conclusion about whether a car is ACTUALLY unreliable or not.

reply

I've been reading Consumer Reports since 1978 in the high school library mostly about the car reviews and reliability. I've been trusting the CR for car's reliability before I shop for cars and it has been very helpful. I switched to digital several years ago. But lately I do feel that their reliability reviews have been very data limited and unconvincing. For example, in this past few days I have been looking at the 2025 Equinox EV with a strong interest to buy. The reliability score by CR was 25 out of 100 and yet you click on the details, it does not have detail data available. Then I clicked on the recall section and it showed 3 items: one was that the brake did not automatically apply hard enough after the system sensed a pedestrian; 2 was the low speed warning sound was not loud enough; and 3rd was the door latch mechanism had some problems. These are all minor and normal problems that you expect in a car with only 1 year of production and they gave the reliability score of 25 which is the lowest of any EV.
Then I looked at the owner's satisfaction: 4/5 and also rated the top among 10 EVs. The owners rated all 4 categories very high: driving experience, comfort, value and styling. I said to myself, if the car is unreliable, there is no way that the owners would want to buy it again (highest rating among 10 EVs. So common sense tells me to not trust the low rating of reliability. I believe that the CR may have used other models like the Bolt's past reliability records to arrive at the rating which is highly unreliable because the Equinox EV's battery system is vastly different from the Bolt.
I also find that the CR's car review is extremely late. It would take them >6 month after purchasing a car to review which is just unacceptable.

reply

I'd say a large part of the issue is limiting the surveys to CR members. Most people love to tell their story to anybody who will listen.
On the topic of the same parts being used in multiple brands, but being perceived as more reliable in some, there could be some truth to that perception. Brands with a reputation for reliability, usually have less features for the money than their competitors and sell well because people believe they're reliable and will last longer. However, people who rank reliability as highly important to them, are also more likely to do regular maintenance. That could be following the maintenance schedule in the owners' manuals, following advise from service people, or things they believe for other reasons. Different manufacturers could also recommend different maintenance intervals than others that share the same part, such as how frequently to change oil or transmission fluid.
We largely don't have solid data that's available to the consumer around those differences either. I do know that a lot of cars are actually very reliable when well maintained, but the demographics that often buy them at different points may defer maintenance or try to do things very cheaply and they become much less reliable. A lot of Toyota and Honda buyers will point out that it is worth spending the money for OE parts, but the 5th or 6th owner of an Audi or Mercedes is often trying to get the cheapest fix they can find.

reply

Thank you for this very in-depth explanation on what’s going on in consumer reports however I would argue as somebody in the car business that consumer report should be on the leading edge of technology and branching out from just their subscription based model and going out and dealing with dealers DMS and warranty claims on new vehicles to see exactly how These vehicles are holding up as they’re out in the world with consumers. I’ve been saying for a couple years now there should be some companies that go back and look at for example how many models are still on the road after 1015 and 20 years just like they used to do in the 90s I think that was a great model and it’s a shame that nobody uses it to advertise. I think if consumer reports really wants to be on the next bleeding edge of giving consumers the information that you can’t get anywhere else they can continue to charge subscription, but again use that money to leverage against warranty claims and dealer DMS to see what type of arrows have been claimed and what repairs have actually been done as far as are they minor are they major Do they take a long time what does it look like I would love to see them come out with a system rating the warranty claim percentages as well as how many arrows per specific model are being done. I feel like this would be a Moore representative data set as well as much easier to get, and you wouldn’t have to pay for it
reply

I can sum up the problem with one vehicle: The tundra. There couldn't have been a bigger upset model year to model year when they retired the old. The old ones sell for brand new prices still years later because of the traditional reliability shift. Went from the million mile tundra to people dumping them before warranty was up due to fear of a blown engine with 0 support from toyota.
Anyways reliability is always a terrible discussion because I can tell you it also depends where you live. Will your car pass your state inspection every year Does your state even have inspections I had to junk a car in NY due to the inability for it to pass emissions inspections, and as a result no one would buy it. In Minnesota, where there's no inspections, I could have driven that car forever and not cared. It had a burnt out wiring harness and O2 sensors didn't work as a result. I didn't feel like paying 3000 dollars in labor to have the harness replaced on a car worth 4000. But if you ask someone in Minnesota, that would still be a perfectly reliable car as it did literally everything else just fine

reply

Many people see cost of maintenance as a part of reliability. For most folk, the importance of reliability is cost of operation when compared to similar models. For instance, higher performance brands, like BMW, often have more work on their maintenance schedules. Valve blasting is one I've seen on BMWs schedules that I've never seen on any other vehicle I've owned. That work also typically much more expensive. On another point, I've been told by many Toyota techs that Toyota customers are much better at getting maintenance done and on time. That plays a major part in long term reliability. It turns out there are different customer cultures in different brands. They can impact the brand positively or negatively. In the case of Toyota, many of their buyers get the PMs done and hold their vehicles long term, I'm sure that lends a strong hand to their perceived reliability. To a BMW owner who keeps their vehicles for 3 to 5 years and gets a new one, those customers will say they're reliable because they didn't have any issues in the short periods they owned the new vehicles.
reply

Looking at the cost of aftermarket warranties by brand would probably tell you a lot about reliability because it will factor in both the likelihood of problems and the cost. Because a catastrophic engine failure will raise the cost of the aftermarket warranty more than car play freezing up sometimes. In 2015 the cost of my aftermarket warranty on my 2015 Jeep Grand Cherokee was $3500 (to extend the warranty to 100, 000 miles. The cost to extend my warranty for my 2023 Lexus ES 300h to 100k miles was only $1750. Exactly half, even though the prices for the two cars are similar. It's also a good metric because I feel like 100k miles is enough wear and tear to give you a good idea how it holds up relatively long term (150k miles would be even more interesting of course. The 3 year surveys are basically useless. Most cars should make it through their factory warranty period these days.
For those curious I did end up pretty much breaking even on my Jeep Grand Cherokee aftermarket warranty. So far no problems on my Lexus ES but it's only 2 years old.

reply

My 2024 Mazda CX-90 PHEV has had what seems like a dozen recalls. All of them were for software updates and none of them affected me in any way that I noticed other than my tailgate needed a shove to close and lock once in a while (also fixed with a software update. I was able to get back and forth and do whatever other daily tasks I needed to do, no stranded at the side of the road breakdowns at all. To me that's reliable with a few new model teething items. Then again I owned cars in the 1970s and some of them didn't start when it rained (or it was fingers crossed exercise) and seeing overheated cars on the side of the road on a hot day was common. To me cars today seem fantastically reliable but to my father 1970s cars seemed fantastically reliable, back when he was young he wouldn't go on a 100 mile trip without good set of tools and an inner tube. Either way I am happy with my Mazda, experiences may vary of course.
reply

Based on my personal experience and experience of my close friends, I don't think Japanese brands are any better than other brands. I have a friend who owns a Toyota Corolla and Subaru Forester since brand new. They are both 10 years old and have problems of rusts, difficulty with repairs, starter problems, fuel trim problems, fading paint and so on.
I had people who owns a Subaru Forester that laughed at me owning an Audi Q5, within a year his supposedly reliable Subaru is dead because of the weak CVT. That happened 8 years ago and guess what - today the Q5 is still soldiering away the miles.
From what I can tell, consumers report attracts customers that thinks Toyota and Hondas are reliable. So rank them subjectively high and this attracts more of those customers. thus creating a tribal group with echo chamber. This bias means their report is self enforcing and is not useful information anymore.

reply

I want a simple, basic car. A 1991 Toyota Corolla. make that a 1993. They had an air bag. 1. 6 4 cylinder, 5 speed manual. Crank windows. Maybe a DX 1. 8 with automatic with overdrive. Those usually had crank windows, but power windows and locks were becoming more expected. Those cars were very reliable. Even those cars are hitting the junk yards 30 years later. Many modern cars are honestly more reliable and better to drive. These Corollas as great as they are, had valve cover, timing belt cover, cam and crank seal leaks. They had timing belts that needed changing, some abused models had engine sludge and burned and leaked oil. Still, a nice well-maintained one could go a lot of trouble-free miles. Today, the Spark, Versa, Rio drivers would blast past one of these old Corollas in a rage of fury.
reply

I hear everything you’re saying and as a retired math teacher, I love numbers much more than the average illiterate math imbecile but if I live long enough to see Chrysler become higher quality and better value cars than Subaru, Hondas and Toyotas despite everything you’re saying is true some companies just care more and are better at making better cars and I don’t think that’s gonna change anytime soon But anything is possible and I’m on my third Subaru and they’re much much better than the garbage domestic brands that I drove before never turned into a survey or maybe once or twice but then a member of consumer reports for 30 years or at least I talk my mom to be becoming a member. I don’t really read it anymore
reply

It would be interesting to see whether the CR new vehicle predicted reliability scores bear up over time when compared to their used car scores. I trust CR, with the caveats you describe. Mercedes and some of these other big makes only fell out of their scoring very recently, but if this tend continues more makes will join them. Mercedes is probably very happy to not be rated, as the last few years they were they were close to the bottom.
For now, the scores are useful for many of the most populsr models, if not all of them. CR members buy mostly very sensible vehicles that much of the population is also interested in.
I'd fill out the CR survey more regularly if we actually drove meaningful miles, but we don't.

reply

Alex you are one of the most valuable resources in the auto industry!
Our 2013 Grand Cherokee Laredo with 249k has been almost flawless.
We had a head gasket fixed last year under lifetime warranty, we had no issues approval. The year before, they fixed a hole in one of the pans.
Last year was our fifth year in a row driving from MI to SC.
During our extended stay (2 months, we made an emergency trip through the ice storm, from Myrtle to Charlotte Airport and back. Pure confidence, but it took more than 4. 5 hours one way. Myrtle beach stayed in emergency for an entire week.
A few weeks ago we went rustic camping in the Northern lower peninsula.
It’s a daily driver.

reply

CR auto reviews and reliability scores have always been suspect to me. Back in the 70s it was said that the CR results were skewed because of over-representation by college professors, who bought a lot of Volvos and usually gave them good scores because they liked them. I suspect the same is happening today with Subaru. Similarly, few of those profs would be caught dead in a Ford or Chevy. I wonder also (but do not know for sure) if/how the results are validated by actual ownership verification. There was a very good reliability score attempt a few years back by someone doing it as a passion project of sorts with hopes likely to monetize it eventually but he gave it up after a few years.
reply

CR buys all the vehicles it tests. CR does not take advertising. CR DOES NOT get flown all over the world by Car Companies with 1st class accomodations and wined, dined and showered with free gifts for several days before given vehicles to test like most of the YT and other car reviewers. JD Power is an advertising agency that is paid for by car companies to make statistics look good for them. JD Power has a very different motus operandi than Consumer Reports. CR may not be perfect but it is still the best info out there for reliability. I recommend taking historic looks and viewing trends of models over several years and not cherry picking certain years or categories.
reply

This is what they have been doing with most of their appliance reviews. LG washer good, new model, yeah it's lg also good. But it has a completely new design, and new inverter pump, yep lg it's good. New samsung bad, because last one was bad. Very few actual reviews and tests done. I didn't fully realize it until they started putting out some tech stuff that I am very informed on, and realized they are just talking out their ass, and realized they might be doing the same thing for most things. I feel like consumer reports now a days isn't to help you make a good decision, just justfiy the decision you already wanted to make.
reply

So funny enough Volkswagen gave me a Consumer Reports subscription and I wonder if that's part of it
I Also got the JD Power survey in my email to respond to (and happily did so)
Another issue with these surveys is bias on the end user. Someone with a problem with a Toyota, who have a reputation for reliability, might not report it honestly but someone with a VW or Audi who have a bad rap might report any minor thing because of the reputation.
Same with Alfa Romeo where most consensus I've seen is that they're fairly solid past the first year and most owners love them (and they're near the top of JD Power now)

reply

I recently had to trade-in my 6 month old 2025 Lexus NX because it rattled like a 1990s econobox. Three trips to the dealership. Sure, it was reliable in that I never had mechanical or electrical issues, but creaks, squeaks, and rattles should carry a higher penalty based on price. I MIGHT put up with creaks and rattles in a sub $20, 000 Nissan or Kia, but it's unacceptable in a $50, 000 Lexus. For that kind of vehicle, I'd consider it a major issue for what you're buying the car for, even if it doesn't affect getting from point A to B. It is definitely something I'd like know when comparing different luxury vehicles.
reply

I have noticed recently that there are sometimes wild swings and a model will go from excellent to fair or poor reliability in just one year, which makes usefulness of the rating questionable. Usually electric cars. For example, the Kia EV6. And now almost every electric car is average or worse reliability, I think the BMW i4 is maybe the only exception but I think that is highly suspect and wouldn't be surprised if it doesn't drop a lot next year. But the i4 is deeply flawed by having lower range for higher price than a Model 3, and so many features are not standard including adaptive cruise.
reply

This is a much-appreciated video. A more important question than reliability is whether the car is under warranty for the issue the car is having. Also how well and how quickly can the problem be solved by the seller of the vehicle. I have found that many auto technicians at car dealers are sadly lacking in skills. At the same time the dealers try to push off a lot of the repair costs back on the buyer of the car. EVs in particular are showing up as not being reliable because mechanics do not know how to work on them, and the owner has to do return trips to fix the same problem.
reply
Add a review, comment






Other channel videos