Consumer Reports (CR) has generally been a fan of the build quality and performance of Apple's computers. However, the consumer watchdog recently cautioned its readers against purchasing Apple's newest MacBook Pro due to discrepancies between Cupertino's published battery life specifications and the figures CR derived in its lab.
Apple and CR were eventually able to pin down and correct a bug related specifically to CR's testing methods, and the nonprofit agreed to rerun its tests. The results of the second round of tests are right in line with Apple's claims. One of CR's test machines even turned a battery life of over 18 hours with the corrections. With the new results in hand, CR now recommends the latest MacBook Pros to consumers.
For the unfamiliar, the issue arose from the fact that Consumer Reports disables browser caching on all machines it tests for battery life. That scenario inadvertently triggered a bug in Safari, which Apple has now fixed. Apple still doesn't believe that CR's testing methods are representative of real-world use, but it seems the sparring partners can at least expect mutually agreeable results from the testing firm's methods for now.
Having had to issue a mea culpa regarding test results ourselves last year, we'd hope this experience leads CR to exercise more caution about presenting wildly variable test numbers as conclusive evidence. Though the initial controversy may have led to a large upswell of interest, it ultimately seems harmful to CR's reputation for rigorous and data-driven testing. Perhaps this fiasco will result in a more conservative approach for the consumer watchdog in the future.