Can you forgive someone who doesn't deserve to be forgiven? If you think about it, those are the only people that you do need to forgive.
If someone deserves to be forgiven, seems to me that you have decided he or she didn't cause you harm or offense after all. So if they haven't offended or harmed you, they didn't deserve your forgiveness. Maybe you misunderstood their intent, or perhaps there were mitigating circumstances that you were not aware of. When you discover these circumstances, and you see the whole picture, you forgive them.
But do they really deserve to be forgiven if they didn't actually harm you? No. If there was nothing to forgive, then you didn't actually forgive them.
So how about those people who don't deserve to be forgiven? The ones who knowingly hurt you, or out-rightly offended you? Does your refusal to forgive cause them harm? Will they suffer in the silence if you never speak to them again?
Probably not. If it was someone who deeply cared for you, they might be hurt by your absence from their life. If the offense and the division breaks up a family, or a marriage, it might serve them right. But will it serve you right?
Will you be better off without that person in your life? Will a divorce, a broken relationship, or cutting someone out of your will make your life better? Perhaps.
But what will you get out of the deal? Smug satisfaction? Freedom? Will you be better off? Or just bitter?