General rule of thumb: As long as you're not increasing voltage, overclocking is "safe" and relatively simple. Keep in mind that most mobos supply auto voltage, so if you OC, your mobo will likely increase the voltage automatically. You'll have to make BIOS adjustments to counteract/prevent this.
Google Search : "[Insert CPU model here] stock voltage overclock"
What processor do you have?
Main: i5-3570K, ASRock Z77 Pro4-M, MSI RX480 8G, 500GB Crucial BX100, 2 TB Samsung EcoGreen F4, 16GB 1600MHz G.Skill @1.25V, EVGA 550-G2, Silverstone PS07B
HTPC: A8-5600K, MSI FM2-A75IA-E53, 4TB Seagate SSHD, 8GB 1866MHz G.Skill, Crosley D-25 Case Mod