Don't waste your time cleaning it, your case is the sole problem; From TR's own Element Q review:
The case's extremely limited airflow and cooler clearance conspire to make it a hostile environment for a Core i3-2100, and that's without even a discrete graphics card installed.
There is no active cooling apart from the PSU - the case is designed for 35W processors using their IGP graphics.
Given that a 65W i3-2100 roasted nearly to death (it definitely throttled) in TR's review, you'd think the A8-5500 would just about cope, right?
Sadly, no. Although an A8-5500 has the same TDP as the i3-2100, the reality is that the i3 will never
use more than 65W, whilst the A8 will probably average 65W under load and peak much
higher. - It's down to the way Intel and AMD define TDP unfortunately
In short, your case isn't going to work for the A8 alone, and the HD6570 will do two things simultaneously:
1) It'll double the amount of heat - the HD6570 uses 64W in addition to the already-too-hot 65W of the A8
2) It'll block off some of the ventilation, ensuring your roast your components even faster than doubling the heat output already will!
To continue using that hardware, you're going to need to run it with the case cover off, or buy another case.
Cheap alternatives are the CoolerMaster Elite 120 or Elite 130, also the Silverstone SG05 and SG06 - both of which have large 120mm fans to blow fresh air into the case.
C10 250 wrote:I'm wondering if changing my processor to an A4-4000 will help?
Nope. Same TDP of 65W; In practice, I expect it to use less power but it'll still run mighty hot in that case, and you still won't be able to run the HD6570.