Yes, this isn't a mouse performance issue at all -- you can verify that by using the context-menu button found on most keyboards -- and it's only happening in Windows Explorer, so you're dealing with a software issue.
JBI and the others are correct -- past occurrences of this have generally been tracked to either a mapped drive that is no longer available or a bad shell extension.
A mapped drive is one somewhere on your network that you connect to either via \\server\sharename or via some drive letter (typically X: or Y: or something else low in alphabet to distinguish it from the local drives in your system). You can do this through the command line (including logon scripts) but the typical method is through an item on the Tools menu in Explorer that leads to
this dialog. These mapped drives can be temporary (lasting only until your next reboot) or "permanent" (in the sense that Windows tries to reconnect to them at your next login). If you have a "permanent" mapping and it is unavailable for whatever reason (or it was temporary in your current session but has gone away, because the other computer got turned off for example), Explorer can get bogged down trying to reconnect to it. This is particularly a problem with the right-mouse-click context menus because 3rd party tools can add their own entries to that menu, and those entries can run code, and that code can be badly-written -- and some of that code runs even before the menu is displayed, because Explorer has to ask each tool whether its entry (or entries) should be enabled or disabled on the menu for the currently selected file.
Which leads to those context menu entries, and whatever programs are associated with them, which can have
all sorts of problems including but not limited to the kind of unresponsiveness you're seeing. That link points you to one
NirSoft utility for diagnosing and fixing general shell extension problems, but in this case I'd point you to a
different one that focuses exclusively on the shell context menu. Unless you recall installing something immediately before this started, you may have a bit of trial and error finding the misbehaving item -- but a good start is turning off everything you don't use and won't miss.
There are a lot of bad shell extensions out there. Generally speaking whenever I install a new piece of software I always choose the "custom install" option if it's available and think very long and hard about enabling any context menu or other shell extension it wants to install (and usually I choose against it -- there very few operations I perform frequently enough directly from Explorer that I want a context menu item for them, and given how crappy so many of them are a little inconvenience of not having the choice on the context menu is outweighed by not having to deal with problems like this.)