I personally always considered sports to be a "muscle" competition primarily. Things like Baseball and Golf throw the definition in for a loop, but one can argue that finesse is a muscle. Even then, the stronger players (ex: Tiger Woods's consistent 300+ Yard drive in his glory days) have a distinct advantage over physically weaker players (ex: Davis Love III, who is more of a finesse / precision player with only a 280 yard drive)
e-Sports are one-step more physical than Chess. Chess is purely mental and I'll never consider it a sport. Its a great game, but not a sport. e-Sports like Starcraft or Counterstrike require pixel-perfect precision on your mouse, and high APM (dexterity in your fingers) to execute basic techniques. Fighting games require timing and precision down to 10 milliseconds (aka: the 1-frame link in Street Fighter).
Nonetheless, that puts e-Sports at the same level of finesse as say, an expert Piano player or Flute player. Marching band, despite the physical exertion, finesse, and significant amount of training involved, is NOT a sport. So I'll never consider e-Sports to be a sport.
With that being said, e-Sports deserve a spot in the national game / entertainment industry. Poker, Mahjong, Starcraft, Chess, Go... all of these games can draw big audiences, big prize pools, and encourage the competitive spirit. If calling them "sports" helps them get recognition, I'm willing to bend the rules for the betterment of their industry.