Consumer Reports retests MacBook Pro after a talk with Apple

For decades, Consumer Reports (CR) has acted as a fairly respected consumer watchdog, testing the claims of countless products and offering up assessments to help the public choose in a world filled with dubious advertising. When the magazine said that Apple's new MacBook Pro was lacking in the battery life department, there was general surprise. After some investigation, though, it appears that the testing conditions were unusual. CR's methodology may have been flawed, though it exposed a Safari bug in the process.

In a statement to TechCrunch, Apple said that CR used a hidden setting in Safari that disabled caching completely. CR explains in an update posted yesterday that the test it ran on the new MacBook Pro is the same one it uses for every other laptop. The lab disables caching in the machine's primary browser and then has it downloading a set of sites over and over again. These conditions were intended to more closely simulate the user going to new webpages over and over, rather than reading the same 10 sites from the cache over and over again.

Apple contends, however, that this method doesn't reflect real-world usage. It also states that "the use of this setting also triggered an obscure and intermittent bug reloading icons which created the inconsistent results in their lab." Apple says it asked CR to run tests with that setting enabled and the Macbook Pro consistently delivers the expected battery life under those conditions. The company also notes that the Safari icon downloading bug has been fixed. CR is now fully re-testing the MacBook Pro and will publish its findings.

The idea of "real-world usage" is a hard one to nail down accurately. Users intermittently consume media, browse, open and close the laptop, and plug it in for charging. While CR didn't want to test re-downloading the same site over and over, many users' browsing habits often mirror that. While I visit plenty of sites throughout a day's work and in my leisure time, the majority of my browsing is done on a tight core of the same sites which, as Apple notes, would take advantage of a browser's caching ability.

As a result of CR and Apple's cooperation, though, readers have a better understanding of what caused the disparity between the official estimate and tested results. Now, only time will tell if Apple's fix to Safari will sort out the battery life issues that end-users are reportedly experiencing.

Comments closed
    • TardOnPC
    • 3 years ago

    “CR explains in an update posted yesterday that the test it ran on the new MacBook Pro is the same one it uses for every other laptop.”

    So they performed a completely new test so Apple can get a better review? How is this OK? People can make the argument the test isn’t real world behavior but it’s not about that. It’s about consistency. You can’t review multiple products if you don’t have a consistent testing methodology.

      • VincentHanna
      • 3 years ago

      They didn’t change their test, they just loaded a beta software patch provided by apple and re-ran the original test.

      Personally, I think they should have waited to update their rating until after the patch was rolled out to gen-pop, but either way, it’s still apples to apples. *rimshot.

    • sophisticles
    • 3 years ago

    I’m obviously in the minority, but I don’t think battery life is that big a deal in a laptop. I own 2, admittedly older, laptops and the only time I’m using battery mode is if I’m sitting in my backyard listening to music or something similar, if I’m using the laptop inside my office or home for anything like I would use a desktop then I plug it in.

    I also fail to see how CR’s testing methodology is considered a valid representation of “real world use”; they set the browser to download new web pages over and over and that’s supposed to mimic how people use their laptops? Really?!?

    I don’t know about anyone else but laptop or desktop or whatever I usually visit about 10 or so websites a day, about 5 tech sites to see if any new tech I’m interested in has been announced or released, 1 or 2 news sites to see what’s going on in the world, 1 or 2 Linux related sites, a video related site and youtube. That’s about it.

    The reality is that you can’t use battery mode for any serious computing anyway, the system is designed to power down to a much lower power state in order to conserve energy, no one is going to encode video, encode audio, edit video, edit audio, play 3d games, compile code, do any serious development work, run VM’s or host a server on a laptop in battery mode (before anyone says anything, yes I once set up a couple of older cheap laptops as web servers to host a small company’s website because they wanted something very cheap and they weren’t even getting 50 hits a day).

      • VincentHanna
      • 3 years ago

      So what you are saying is that you spent a bunch of extra money on laptop parts vs desktop parts (and probably made other tradeoffs like processor power, ram speed, etc) and you basically just use your laptop as if it were a desktop anyway, and therefore who cares if this laptop sucks at being a laptop, because it does okay at desktoppy stuff (admittedly not as well as a desktop of equal price, but whatever!)

      Sadly, I don’t believe you are in the minority at all. People are idiots.

      [quote<] I also fail to see how CR's testing methodology is considered a valid representation of "real world use" [/quote<] It's not. Its meant to stress the components to a specific, pre-defined level to create a standardized baseline on which to make relevant comparisons across brands and test battery life claims. [quote<]The reality is that you can't use battery mode for any serious computing anyway, the system is designed to power down to a much lower power state in order to conserve energy, no one is going to encode video, encode audio... ... ... etc[/quote<] Which was not part of the test in question. The test in question was web browsing, which you already admitted you do do. Unfortunately, your test, which was visiting 5-10 websites and downloading a cat video isn't a very good test. Which is presumably why you don't get paid to test laptops.

    • End User
    • 3 years ago

    Ugh. Consumer Reports is now stating “New Apple software fixes a battery issue found in CR tests”. There was no issue with the battery. There was a bug in a developer mode featured in Safari. The Safari bug has been fixed by Apple. A recent software update supplied to Consumer Reports resolved the problem they encountered in the earlier testing.

    [url<]http://www.consumerreports.org/apple/consumer-reports-now-recommends-macbook-pros/[/url<]

      • VincentHanna
      • 3 years ago

      You are right.

      The issue was with the laptop’s battery life, not it’s battery.

      Thank you Grammar Police.

        • End User
        • 3 years ago

        Battery life is a separate issue.

    • Klimax
    • 3 years ago

    A note on caching:
    For most part vast majority of final HTML files have caching disabled altogether to prevent old content to show. (Getting it right with caching permitted is hard and won’t with modern dynamic sites get much better)

    When it comes to other files like JavaScript, CSS or images it depends on what they serve and how well are Caching directives implemented. (If at all) If they are ads or tracking files, then expect cache to be severely limited. (At best there will conditional request) If it is part of content presentation then usually it will permitted to be cached and often there will be no request whatsoever. (Assuming directives are set correctly)

    Unfortunately many sites don’t get it right and performance suffers. The worst offender know to me is Tumblr where each image server has different caching directives with no logic (one will permit caching only for 24 hours while the other for several months) and there is no ETAG for conditional request/validation leading to excessive bandwidth usage. (DeviantArt got it mostly right BTW)

    In short, their methodology is actually quite closer to real situation then many suspect.

    • ronch
    • 3 years ago

    CR is nuts. How can an Apple product ever have any issues? It’s an impossibility.

    • Krogoth
    • 3 years ago

    The amount of salt from Macoytles and apologists is delicious.

      • Pwnstar
      • 3 years ago

      Fanboys are funny.

      • tipoo
      • 3 years ago

      Is it…Impressive?

        • Krogoth
        • 3 years ago

        [url<]https://youtu.be/uvUL28Skt6E[/url<]

    • DPete27
    • 3 years ago

    How many web pages will a browser (Safari) actually cache before it flushes old ones though? Is there ever a time limit for cached web pages to be held?

      • Voldenuit
      • 3 years ago

      How much cache could a webcache cache if a webcache could cache cache?

      CR: A webcache could cache no amount of cache since a webcache can’t cache cache.

      Apple: A webcache could cache cache if a webcache would cache cache.

      TR: Even if a webcache could cache cache, and even if a webcache would cache cache, [i<]should[/i<] a webcache cache cache? Anon: A webcache should cache cache if a webcache would cache cache if a webcache could cache cache.

      • Klimax
      • 3 years ago

      Web pages themselves are not cached at all, because Cache directives forbid it. (Unless fully static or quite rarely regenerated or cache directives are perfectly set – usually leading to effectively same effect on cache)

      Only associated files like JavaScript or images are generally permitted to be cached.

    • blastdoor
    • 3 years ago

    One could pick nits on either side, but I think CR and Apple both handled this reasonably well.

      • adisor19
      • 3 years ago

      Nah, this was poorly handled by both CR and Apple.

      CR used the failed results for a bad recommendation without further investigation and Apple went on the defensive more or less accusing CR or improperly testing their laptop if you read between the lines.

      IMO, CR should have questioned and investigated the obviously bad results and Apple should have accepted the bug, apologized for it and shut up about it.

      Adi

        • Glorious
        • 3 years ago

        Complete nonsense. Will you ever read the report?

      • chuckula
      • 3 years ago

      Stop trying to straddle the fence. Adisor was much more flameworthy!

    • tipoo
    • 3 years ago

    This bug fix then won’t do anything for home users complaining about the battery life, as it’s specific to the hidden dev menu and disabling caching.

    I think the battery life just is what it is at this point, the hardware they chose painted them into a corner. They cut 25% of their battery capacity with 75Wh now in the top end, and relied on Skylake and Polaris’s deeper idle states to save the power. The problem is, the minimum power went down, but when a user is actually doing something, the allowable power is similar to before (except the GPU TDP shrinking). So more demanding users will probably always see a regression from last gen for the extent of its life, while users that allow a lot of idle time see an improvement.

    75W battery, 7W low load, 10ish hours battery life.
    99Wh battery life, 10W (where I am) typical load, 10ish hours battery life.

    But the TDP of the CPU being the same, 45W drawing from a smaller battery obviously has a disproportionate impact.

    I keep going back to…This years components on last years battery size would have been awesome. I also get the feeling the design choices were made with Kaby Lake in mind.

      • adisor19
      • 3 years ago

      Yeah, this is more likely the reality and while the lower bat life is a side effect it is in no way comparable to what CR published.

      Adi

      • blastdoor
      • 3 years ago

      I guess the consolation for people who buy the MBP is that at least Apple didn’t try cramming the old battery into the smaller case, resulting in exploding laptops.

      • davidbowser
      • 3 years ago

      Agreed. The CR thing was essentially a sideshow for people that are having battery problems.

      I have too many Apple products in my house to dismiss these things out of hand, and the Apple support forums usually bring some sophisticated troubleshooting (programmers and engineers) from the community. If there are enough of them posting bug reports, then it is real.

      Related note: I ran into the 4k monitor power save bug that required shutting down the Mac completely to get it to recognize the monitor again because waking from sleep didn’t send the video signal back. It took MONTHS and 2 point releases until that support thread started to die out. There were people on there that had returned their monitor and/or their mac only to see the problem happen again.

      • End User
      • 3 years ago

      Battery life 2015 vs 2016 is a mixed bag:

      [url=http://www.notebookcheck.net/Apple-MacBook-Pro-15-Late-2016-2-6-GHz-i7-Notebook-Review.185254.0.html<][quote<]We are a bit surprised that the smaller battery capacity was compensated that well. The runtime does take a hit when you really stress the system, but Apple was actually able to improve the runtimes in the practical tests.[/quote<][/url<] The issue I have is it maxing out at 16GB and no Vega/Pascal.

        • tipoo
        • 3 years ago

        Cuda is faster than OpenCL on AMD is faster than OpenCL on Nvidia, on roughly analogous hardware, which leads to the situation where Premiere is faster than FCPX on Nvidia while FCPX is faster on AMD than Nvidia…I think Nvidia will have to reconsider their OpenCL priority if they want to be in Macs again. As for Cuda Apple probably doesn’t want to tie themselves in to a single vendor standard.

        And Vega won’t be out for 6 months for anyone, so that was a non-starter. I am however bemused that a mere two months after the MBPs launched, the XPS 15 with the direct Kaby Lake replacement of its CPU launched, so idk why they wouldn’t have just waited for that.

          • End User
          • 3 years ago

          I won’t get a replacement for my work supplied 15″ MBP until Q1 2018. Hopefully Apple will have come out with a refresh of the 15″ MBP by then.

      • ptsant
      • 3 years ago

      They cut battery capacity because … look how thin it is!
      They also could not fit a type A USB because … thin.
      They also changed the excellent keyboard of the previous macbook because … thin.

      Does one actually have to study industrial design or it is simply about making stuff that is thin?

        • tipoo
        • 3 years ago

        Le sigh. Imagining a modern MBP that reintroduces the original unibody thickness with the 70W GPU TDP that allowed (might have only been in the 17″, but at least 50W), number of ports, and battery size + the RAM that could power, makes me drool, but Apple will never go back that far on thickness.

          • Chrispy_
          • 3 years ago

          No you crazy mad fool. Begone before we all lynch you for being stupid!

          [b<]Who in their [i<]right mind[/i<] would want a laptop with a large array of useful ports, a keyboard that is nice to type on, all day WORKING battery life or CPU and GPU performance that professionals use?[/b<] I think you're misunderstanding what the "Pro" means in Macbook Pro; Silly tipoo, t's short for [i<]Profile[/i<] not "Professional". You're only supposed to use it to run Facebook so that you can "like" Tim Cook on Facebook and upvote Apple in various discussions around the web. Easy mistake to make, but I hope I've cleared up that little misunderstanding now.

    • PBCrunch
    • 3 years ago

    Both parties are in the wrong here. Apple’s software is buggy. Consumer Reports should have a more robust test methodology than turning off caching, which isn’t a very realistic test scenario. Perhaps their testing should involve something like a server generating a mix of static content and random pages that can’t be cached.

      • Glorious
      • 3 years ago

      Let me get this straight: Consumer Reports is in the wrong here because they helped uncover one of Apple’s bugs?

      Why don’t you guys start your own magazine, called Producer Reports, which is intended to show manufacturers in the best possible light?

        • PBCrunch
        • 3 years ago

        CR is partially wrong because disabling all caching is not typical browser use. A mix of novel and cached assets is more representative of the way people actually use computers. The testing should probably reflect that.

        • adisor19
        • 3 years ago

        Nothing wrong with uncovering a bug. One would normally report bug to manufacturer, inform readers they most likely discovered a bug and based on these findings is a recommendation or not.

        CR did not contact the manufacturer. CR did not conclude the crazy bat test results were due to a bug. They simply published the crap results with no questioning and issued a non recommendation of the product.

        Adi

          • Glorious
          • 3 years ago

          False.

          You didn’t read it.

            • willmore
            • 3 years ago

            Why does this make me flash back to A Fish Called Wanda?
            “Otto West: Apes don’t read philosophy.

            Wanda: Yes they do, Otto. They just don’t understand it. Now let me correct you on a couple of things, OK? Aristotle was not Belgian. The central message of Buddhism is not “Every man for himself.” And the London Underground is not a political movement. Those are all mistakes, Otto. I looked them up. ”

            Maybe Adi did read it but just didn’t understand it?

    • backwoods357
    • 3 years ago

    It’s not like the battery is the only issue with these guys. Apple has lost the cohesive strategy that users came to expect. Jobs was a dick, but he knew what people would buy and was very concerned about the overall polish of a product. Without his vision it isn’t surprising that customers are walking away and the stock is dropping.

    • Voldenuit
    • 3 years ago

    Meanwhile, Mac market share is at a [url=http://www.digitaltrends.com/apple/mac-market-share-drops-to-five-year-low/<]5 year low[/url<]. I'm sure this is just a sampling error and they should have enabled cached results.

      • blastdoor
      • 3 years ago

      I am definitely a Mac fan, but this isn’t at all surprising.

      It’s really disappointing how poorly Apple has done with respect to updating the Mac over the last few years.

      I contend that this will ultimately be bad for the iPhone, too. Every Apple product is more valuable when combined with other Apple products. The Mac might be lower volume, but the installed base is nothing to sneeze at — about 80 million or so. I’ll bet a very large fraction of those 80 million also own iPhones. If people start leaving the Mac, their attachment to the iPhone will diminish.

        • Klimax
        • 3 years ago

        Actually it is even worse for Apple. Since they don’t permit IOS development without Mac, should they lose enough share there they can start losing mobile too.
        (With remaining devs using shared Macs with virtualization for it)

      • tipoo
      • 3 years ago

      I really like macOS, but I’m not surprised. Windows laptop hardware has come a long way in the last 5 years, and Apple keeps doing things the core professional market doesn’t particularly want (I don’t care if it’s 3mm thinner, I want a 50W GPU, 99Wh battery, and the RAM that larger battery can power!), the iPhone being their favorite child and cash cow now. Macs are only 10% of their revenue.

        • End User
        • 3 years ago

        [quote<]Macs are only 10% of their revenue.[/quote<] Only.

          • tipoo
          • 3 years ago

          Yes? If the iPhone suddenly tanked, they’d be boned, so guess where they afford more of their attention.

            • End User
            • 3 years ago

            If you remove iPhone revenue Apple is still in the top 50 on the Fortune 500 (using revenue as the measuring stick).

            [url<]http://beta.fortune.com/fortune500/list[/url<]

            • tipoo
            • 3 years ago

            I don’t disagree that every part of Apple is massive, even the Watch dwarfed many well established watch brands in year one. My point was, the 80% of their revenue no doubt gets more love than the 10% of their revenue.

            I mean, give this a read, Gurmans sources are usually dead on.

            [url<]https://www.bloomberg.com/news/articles/2016-12-20/how-apple-alienated-mac-loyalists[/url<]

            • End User
            • 3 years ago

            The alienation started a decade ago or more. I felt it when I bought my Power Mac G5 2.7 in 2005. The replacement (2007’ish) for the G5 was a Q6600 box running Linux as the alienation had set in by then in a big way. There was a brief respite in 2012 with the fantastic quad core Mac mini. Unfortunately that did not last and we have now slipped even deeper into the bucket of Mac despair.

          • Glorious
          • 3 years ago

          I had to double check their 10k because I had trouble believing it was even that much.

          [url<]http://investor.apple.com/secfiling.cfm?filingID=1628280-16-20309&CIK=320193[/url<] But, haha yeah, it's "Only." [i<]twenty-two billion.[/i<]

            • End User
            • 3 years ago

            The Mac product line on its own would be #124 in revenue on the Fortune 500 (double check though I may be wrong):

            [url<]http://beta.fortune.com/fortune500/list[/url<]

      • davidbowser
      • 3 years ago

      This is certainly something that Apple needs to work on. The hardcore Apple folks (I favor Apple, but am pretty agnostic) have been getting restless about the lack of updates to the Mac Pro, Mac Mini, and the less than stellar reception of the recent Macbook Pro (which is not Pro enough for almost anyone).

      That said, the source numbers for that reference were statistically noisy, to the tune of month to month gains or drops of over 1%. Not year to year movement, but month to month. So that would be equivalent to say that suddenly several million people were using a Mac and then changed to something else the next month, and then changed back again.

      • Krogoth
      • 3 years ago

      It is because they are going back to mid and late 1990-era in the PC market. There are too many “clones” that are flooding the market. Aside, from the brand name. There isn’t really anything special about Mac upline versus other ultra-portables and all-in-one units that are on the market.

      You can see why Apple’s shareholders have pretty much abandon their traditional market in favor of eGadgets.

    • adisor19
    • 3 years ago

    Their results were way off the charts and should have never been published in the first place. CR should have contacted Apple and confirmed the anomaly before loosing credibility.

    This is more like it but the damage has already been done at this point for both Apple and for CR. CR lost credibility and Apple lost a tad more confidence in their new product.

    Adi

      • derFunkenstein
      • 3 years ago

      So they should never have reported that they’re getting poor, hugely-varying results and instead hide inconvenient facts for Apple’s sake?

        • adisor19
        • 3 years ago

        No, they should have reported their incredibly messed up results to Apple and only after hearing from them, they should have taken a decision on whether to publish the results as is, retest with fixed software, etc.

        Publishing BS results without contacting Apple in the first place to at least find out why their results are abnormal, reeks of amateurism.

        Adi

          • derFunkenstein
          • 3 years ago

          Oh, good idea. Let’s keep everything a secret. /s

            • Glorious
            • 3 years ago

            Yup, and get special “fixed” software to “resolve” problems that their test suite has.

            There’s no way a company could abuse that! They couldn’t possibly lie to a magazine and send software that detects a test suite and cheats.

            No, they’re definitely more scared of a magazine than the government of the United States, who just arrested a VW executive for doing exactly that.

            • adisor19
            • 3 years ago

            Again, that’s not what I’m saying. They should have contacted Apple first and say that they encountered some weid results with their test and that they can’t issue a conclusion right now OR at least wait to see what Apple has to say before issuing a do not buy.

            Adi

            • Glorious
            • 3 years ago

            They did contact Apple. You didn’t even read it.

            [url<]http://www.consumerreports.org/laptops/macbook-pros-fail-to-earn-consumer-reports-recommendation/[/url<] "Apple declined to comment on our test results until they better understand the issue" The product is being sold right now, if they sit on bad results until a company decides to respond that is literally giving product manufacturers prior restraint.

            • derFunkenstein
            • 3 years ago

            You sound like you’re saying they should have not published these results. “Let’s keep it secret and work it out with Apple for however long it takes for Apple to get it right” is a disservice to the readership that apparently expects CR to weigh in on the new designs. It’s ridiculous to suggest that Apple should get special treatment. I wouldn’t expect them to give extra mercies to HP or Dell or Lenovo, either.

            • Glorious
            • 3 years ago

            He’s either saying CR should do WHAT IT DID, or he’s saying that manufacturers should literally have prior restraint over reviews.

            Either way, he’s an impressive specimen of fanboy.

            I’m surprised he even reads techreport.com. Why not just get your “reviews” straight from Apple.com?

            • kamikaziechameleon
            • 3 years ago

            If you look into the matter more you’ll discover they did reach out to apple.

            • UberGerbil
            • 3 years ago

            I’m sure Volkswagen wishes things worked that way.

          • Glorious
          • 3 years ago

          This reeks of unabashed fanboyism.

          1) Consumer reports, crazily enough, reports to consumers.
          2) *THEIR* incredibly messed up results? Oh, so it was *THEIR* bug that generated them?
          3) When I buy a laptop off the shelf, LIKE CR DID, do I get to retest with this fixed software?

          Apple admits the bug, why won’t you?

          • kamikaziechameleon
          • 3 years ago

          They actually did reach out to apple prior to publishing.

          • Inkling
          • 3 years ago

          [quote<]Publishing BS results without contacting Apple in the first place to at least find out why their results are abnormal, reeks of amateurism.[/quote<] Sad fact is that not every publication can be like TR.

        • FastGoose52
        • 3 years ago

        That sounds a lot like… Fake News

        • slate0
        • 3 years ago

        You and your voters are being idiots. In the real science and engineering testing world, when something comes out super weird, you check your own processes first, and damn right you talk to the manufacturer about it before you go publishing nonsensical data.

          • derFunkenstein
          • 3 years ago

          They did, if you read the report. Apple had no comment because they didn’t know the cause yet, so CR went public. This is the same reading-comprehension problem Adisor had.

      • Glorious
      • 3 years ago

      They stuck to their methodology. The same methodology they use for all other laptops.

      Doing *ANYTHING ELSE* would damage their credibility.

        • adisor19
        • 3 years ago

        Nothing wrong with sticking to their methodology for better or for worse. What is wrong is for them to publish crap results when clearly something is terribly wrong with the results. They should have investigated the WHY of the results first.

        Adi

          • willyolio
          • 3 years ago

          there was nothing wrong with their results. There was something wrong with the product. Which they find on a regular basis across all types of products.

          Phoning up the corporation when their product doesn’t live up to marketing so they can request an extra-special unit or instructions on how to use it differently from everything else just so they can get the claimed results?

          lol. You might as well just depend entirely on advertisements for your information.

            • adisor19
            • 3 years ago

            What are you talking about ? There was something wrong with their results. You might wanna go check it out again. Their results were off the charts. The least they could have done is say : something is very wrong with our battery life results, we have contacted manufacturer for comments and for the time being do not buy.

            All that CR did is say : results are these, do not buy.

            Adi

            • Glorious
            • 3 years ago

            YOU DIDN’T READ IT. AT ALL. STOP SPREADING MISINFORMATION.

            THEY DID EXACTLY WHAT YOU SUGGEST:

            “The least they could have done is say : something is very wrong with our battery life results, we have contacted manufacturer for comments and for the time being do not buy.”

            [url<]http://www.consumerreports.org/laptops/macbook-pros-fail-to-earn-consumer-reports-recommendation/[/url<] "Apple declined to comment on our test results until they better understand the issue"

            • willyolio
            • 3 years ago

            of course he didn’t read it, he doesn’t want to taint his perfect vision of apple with potentially negative information.

            • derFunkenstein
            • 3 years ago

            hehehehe you said “taint”

            • Pwnstar
            • 3 years ago

            heh heh hehe.

            • Goty
            • 3 years ago

            There was something wrong with their results? Then why was it Apple and not Consumer Reports that had to fix something before the device was to be retested?

          • Glorious
          • 3 years ago

          Nothing was wrong with the results. APPLE EVEN ADMITS THIS: They had a bug!

          Why can’t you?

            • adisor19
            • 3 years ago

            I think you’re confusing TESTING METHODOLOGY with RESULTS (please forgive me oh mighty sweatshopking). The methodology was sounds. The results were crap. Get it ?

            Adi

            • Glorious
            • 3 years ago

            Yes, I get it: You didn’t even read the article and yet you are spreading misinformation.

          • kamikaziechameleon
          • 3 years ago

          I’m seeing a pattern here. You might have wanted to investigate the topic before posting your opinion. 😛

        • ImSpartacus
        • 3 years ago

        They should’ve released all results. Both those under their tried & true methodology and those under a modified methodology after going back & forth with Apple. And of course, they need to include a comprehensive write up explaining the bug and why they included a modified methodology. THAT is transparency.

        When you “stick to your guns”, you can look bad. The kind of tunnel vision that cr demonstrated was immature and unhelpful.

      • kamikaziechameleon
      • 3 years ago

      The test was valid and the major issue was a glitch created by apples software not CR. Also Apple’s quoted expected run time is far from what actual users are experiencing. So this is all laid at apples feet. Not that the product is bad, just a bad launch. There are so many moving parts in a computer ecosystem.

      The irony of it all is how apple has promoted itself as perfect and has been getting more wonky with age. Meanwhile despite many horrible product gaffs Microsoft is getting really good at smoothing out their ecosystem dramatically… arguably surpassing apple in this regard.

        • adisor19
        • 3 years ago

        Yes the test was valid , but the results of the test were not. First of all, the test they did changes a bunch of defaults that users would NEVER encounter as nobody but web dev would actually go ahead and turn off Safari catching in the first place so one could argue that the methodology of their test is flawed. I’m not gonna argue that however. What I’m arguing is that they saw crazy results from their tests and instead of contacting the manufacturer for an explanation, they went ahead and published the results together withe NOT recommended buy for consumers.

        This is sad because 99.999% of consumers would never encounter this bug in the first place and thus would never have such low battery life as depicted by CR.

        Again, I have nothing against their methodology. What bugs me is that they went ahead and published BS results and a bad recommendation instead of investigating why their results were so crazy in the first place.

        Adi

          • Glorious
          • 3 years ago

          “but the results of the test were not.” BECAUSE THEY REVEALED AN APPLE BUG?

            • adisor19
            • 3 years ago

            No, because they were an anomaly. Bug or not, the results should have been investigated first.

            Adi

            • Glorious
            • 3 years ago

            They reported the anomaly. Until it was resolved, as they said they reported it Apple, yes, do not buy.

            They followed what you recommend, you don’t know this because you never actually read it.

      • emorgoch
      • 3 years ago

      I have to completely disagree, and from my perspective, it only increases CR’s credibility. Their whole testing model it “what a standard consumer can expect”. This is why they go and buy units from retail rather than getting samples direct from the manufacturers.

      As a standard consumer, I wouldn’t have visibility from Apple to get a team of engineers to my house to see how I’m working, and look at my system to do hands-on diagnostics. At best, I’d get a standard line from tech support stating that “many factors can affect battery life,” etc.

      As for running with or without caching: Their web test loads only 10 different static pages from their internal web server on repeat. If caching was turned on, it wouldn’t need to ever load any more data after 30 seconds, which would drastically reduce the “real world” scenarios that most people see. Disabling the cache makes it more reflective of how people actually browse the web, as you’re constantly viewing different web sites that each have their own components that need to be downloaded.

      *Edit: What I’m really finding interesting in all this is how much the media is playing up the disabled cache aspect of it, vs. the fact that THERE’S A HUGE BUG IN APPLE’S SOFTWARE THAT’S DRAINING THE BATTERY LIKE MAD WHEN THE CACHE ISN’T USED. I’d expect this line to be the standard line from Apple, because they always downplay and deflect any issues with their products, but I was expecting CR’s follow up to basically put it front and center to say “Apple found a bug with their software that’s causing our inconsistent results. If/when they release a fix, we’ll re-run our tests and update our findings, but until that time, these laptops suck.”

      • superjawes
      • 3 years ago

      [quote=”adisor19″<]CR should have contacted Apple and confirmed the anomaly before [u<]loosing[/u<] credibility.[/quote<] Using "loosing" instead of "losing" is quite ironic in this case 😉

        • adisor19
        • 3 years ago

        Gonna leave that unedited for the lulz 🙂

        Adi

      • Goty
      • 3 years ago

      [quote<]Their results were way off the charts and should have never been published in the first place. [/quote<] Nevermind that people who actually bought the laptops reported the same thing.

        • Voldenuit
        • 3 years ago

        They were using it wrong.

        • adisor19
        • 3 years ago

        No, they didn’t. Nobody got the crazy variation that CR reported.

        In general, bat life was indeed lower but nothing to the point of CR’s results.

          • Goty
          • 3 years ago

          You have an amazing internal filter, don’t you?

        • NeelyCam
        • 3 years ago

        Did they…? I haven’t seen those reports.

        What I recall from Apple’s commentary on the issue, CR was using a developer mode that triggered a bug in caching. Normal mode (what regular people would be using) didn’t have that bug, which is why regular people didn’t see this battery life discrepancy..

        Please correct me if I’m wrong, but that’s how I remember it.

      • Wirko
      • 3 years ago

      If CR decides to also re-test some competitive notebooks with caching enabled then the only remaining loser will be Apple.

      • flip-mode
      • 3 years ago

      It’s very sad Adi. I’m sorry this has hurt you so deeply. Maybe you can request a free year of Consumer Reports.

        • adisor19
        • 3 years ago

        Already did. Can’t wait!

        Adi

      • thedosbox
      • 3 years ago

      [quote<]CR lost credibility[/quote<] No, you've lost any sort of credibility with this comment.

        • adisor19
        • 3 years ago

        And why is that ?

        Adi

        • willyolio
        • 3 years ago

        eh, adi has been our resident apple fanboy for years. he’s never had any credibility or objectivity when it comes to anything involving apple anyway.

      • tipoo
      • 3 years ago

      If a car accelerated uncontrollably and CR couldn’t diagnose it, would they not publish it? It’s an extreme example, but a buggy test case is still something to report, that’s their job.

      Also not publishing it may not have gotten a bug fix out this fast.

        • adisor19
        • 3 years ago

        Nothing wrong with publishing it but they didn’t bother to investigate whether it was a bug or not.

        Based on your car analogy, it’s more like fuel consumption : Apple says 5l/100km while CR’s weird test finds results varying between 6l/100km and 25l/100km. CR goes ahead and publishes those results with no questioning and issuing a no buy advice.

        If I were CR, those results would scream at me that something is wrong somewhere but, hey, I’m not CR.

        Adi

          • tipoo
          • 3 years ago

          They made it pretty clear it was odd behavior and they were reaching out to Apple to find out why, as well as publishing what they did get. That would still be the appropriate move in the car analogy.

            • Glorious
            • 3 years ago

            Nothing is clear unless you actually read it.

            • Pwnstar
            • 3 years ago

            lmao

      • TheRazorsEdge
      • 3 years ago

      Consumer Reports published what they saw. As this was due to an obscure bug, Apple probably had no definitive explanation at the time.

      If Apple had identified a likely cause and promised a fix, then I would expect a statement to that effect in the review.

      But a poor result with no explanation? Let everybody know, and they can decide for themselves. Especially if the test methodology is available so people can judge how relevant the data are.

      • AnotherReader
      • 3 years ago

      May the RDF be with you

      • gmskking
      • 3 years ago

      I am just commenting to say that I think TR should remove the negativity with this site by removing the thumbs down option. Really, we are mature adults and we do not need a thumbs down option. Lets keep it positive. 🙂

        • Klimax
        • 3 years ago

        I am with you if remove voting completely! ETA: Let’s be fair there…

      • LocalCitizen
      • 3 years ago

      i think what Adi is saying is that even tho CR made the correct conclusion (macbook pro’s battery life sux), CR’s testing procedure might have encountered a bug that is not common to most users (i clear my web cache often for privacy reasons, so i would be the type who would run into this … if i used safari)

      the question is would this hurt CR’s reputation? my argument is it won’t, because the conclusion matches what people see on most websites.

      could CR have contacted apple before publishing the result? i feel it probably has, but was probably not taken seriously. so what do you do if you are CR? you publish the results, and force apple to respond. just like what it has done to the truck / suv stability problems some 10+ years ago.

      what can we expect from the retest? a more consistent battery life . one that may even meet apple’s claims. would CR’s reputation suffer then? probably not. they could just say apple made a fix, and the issue is gone, which is true.

      • ptsant
      • 3 years ago

      Whether one should test with or without cache can be debated. If you don’t think it relates to your use patterns, don’t consider these tests in your purchase decisions. I don’t consider tests with Assassin’s Creed when I buy a GPU, because I never play Assassin’s Creed.

      What cannot be debated is that the notebook must work as expected. There was a bug. The bug was Apple’s fault and has been corrected. Lesson learned, hopefully.

      • Waco
      • 3 years ago

      Reading before posting should be required. You clearly didn’t read the articles at all.

    • nico1982
    • 3 years ago

    One of the issue was inconsistency in runtimes, something the cache being disabled wouldn’t explain. Au contrair, it should make the test more deterministic, I think.

    As a side note, I noticed that the CPU – i7-6700HQ – idles at 1.3 GHz on OSX and 1.0 GHz on Windows. It might have a part in the lower battery life (battery capacity aside).

      • cygnus1
      • 3 years ago

      All things being equal (meaning no bugs to muck things up), the cache being disabled allows them much more repeatability across devices and browsers. The inconsistency was actually blamed on the Safari bug, which was only able to cause problems while the cache was disabled. I don’t believe the blame for inconsistency was ever explicitly put on just having the cache disabled. They were very aware that disabling the cache was causing unexpected behavior, and that’s why they went to Apple to see what was up.

    • TheRazorsEdge
    • 3 years ago

    A test doesn’t have to reflect real-world usage in order to be meaningful or useful.

    In fact, with all of the settings that may affect battery, it is necessary to use a very detailed and standardized procedure. If this exposes bugs or shortcuts, so much the better.

    That said, it would also be good to do a basic test with default settings, as that is what most users will do.

    I believe a good review includes both standard use scenarios and rigorous, methodical testing.

Pin It on Pinterest

Share This