Your processor determines the maximum memory quantity and speeds, since the memory controller has been integrated into the processor for around the last decade of CPUs.
If you tell us what CPU and motherboard you're using, we can tell you what's officially supported, as well as what is likely to work.
A pair of 4GB DDR3 sticks is worth about $40 and a pair of 8GB DDR3 sticks is worth about $50-60. I'd be tempted to buy all-new RAM so that you have 4x8GB DDR3 1600, and dump the old stuff on ebay or craigslist. Technically you can mix and match RAM speeds, timings, and voltages because of the JEDEC spec - there will always be a common ground that different RAM modules will run at. The problem is that the only common timings shared by both different sets may be super-slow (like DDR3-1066MHz). I've run mixed size/timings in a few different machines before and had no problems but there are definitely more caveats, asterisks, and fine print to read if you do.
You can definitely mix and match, but unless you've stumbled across a 2x8GB kit for free, or silly-cheap, just replace the whole lot so that it's matched. You'll be able to run at the RAM's rated speed then, as long as the processor actually supports that speed - and older DDR3 processors like Ivy Bridge (e.g. i7-3770) and Haswell (e.g. i7-4770) actually benefit significantly from faster RAM.
Congratulations, you've noticed that this year's signature is based on outdated internet memes; CLICK HERE NOW to experience this unforgettable phenomenon. This sentence is just filler and as irrelevant as my signature.