I agree that it's subjective. Certainly someone wouldn't expect to get less FPS on a cartoon-art game than a photo-realistic one (assuming action and complexity are similar). In that regard, I think it does put things into perspective when we occasionally get games that seem to look "ahead of their time" AND run pretty well on most hardware. Comparing that to a game that may produce 2/3 the FPS on the same hardware because they were pushing ray-tracing (for example) to get a marginally more realistic feel can make people feel like the game devs failed to "optimize" their game.
Few/None of my top-rated games got that acclaim solely because they looked pretty. Sure, visuals can go a long way to influence your immersion in a game, but there is definitely a point of diminishing return.
Main: i5-3570K, ASRock Z77 Pro4-M, MSI RX480 8G, 500GB Crucial BX100, 2 TB Samsung EcoGreen F4, 16GB 1600MHz G.Skill @1.25V, EVGA 550-G2, Silverstone PS07B
HTPC: A8-5600K, MSI FM2-A75IA-E53, 4TB Seagate SSHD, 8GB 1866MHz G.Skill, Crosley D-25 Case Mod