My understanding from having discussed this with Catholics is that the priest does not himself forgive sins, but acts as a conduit of God's forgiveness. I don't know that I buy that either, but I do also see the cathartic value in confessing one's sins to another.
It's all something I still have trouble with. All these denominations and branches of Christianity claim they're the only right one. It seemed ridiculous to me when I was younger but as I get older I start having more of those "don't gamble with your soul" thoughts and want to know which direction to go.