First, you need to put in a quality surgical stainless steel, niobium or titanium CBR in the piercing but do not do it yourself. Buy the jewelry at your professional piercing studio (tattoo/piercing shop, NOT Claire's or Piercing Pagoda) and they will help you find the right size and put it in for you for free. Then, you will need to leave your CBR in for several weeks - no changing the jewelry until it is completely healed. Second, you need to clean your piercing properly and do about 3 sea salt soaks every day until the bump is completely gone. Cleansing and sea salt soak instructions are here.
Once your piercing is completely healed, you can change the jewelry - again, use high quality jewelry only from a professional shop and let them put it in for you so you don't accidentally damage anything in the process. Good luck to you - I'm sure your piercing will be just fine. My nose did exactly the same thing and this happens all the time. If at any time it becomes infected - painful, red, hot, pussing green ooze - then you need to see a doctor. Otherwise, these instructions should help you heal up just fine.